From kits of parts to an autonomous fleet
in 857 easy steps
without hiding anything
The last version of this book and other documents are available at the URL
http://book.duckietown.org/
For searching, see the one-page version.
DB17-jwd)DB17-w
DB17-j
DB17-d
DB17)
DB17)
DB17-jwd)
DB17-wjd TTIC)DB17-w
DB17-j
DB17-d
rviz
DB17-lc configurations
ssh-keygen
rqt_console
roslaunch
rviz
rostopic
catkin_make
rosrun
rostest
rospack
rosparam
rosdep
roswtf
rosbag
roscore
pkg_name
duckieteam
duckietown
duckietown_msgs
easy_algo
easy_logsdownload
easy_nodeeasy_node API
easy_node: the userβs point of view
easy_regression
what_the_duck
Duckietown is a robotics education and outreach effort.
The most tangible goal of the project is to provide a low-cost educational platform for learning about autonomous systems, consisting of lectures and other learning material, the Duckiebot autonomous robots, and the Duckietowns, which constitute the infrastructure in which the Duckiebots navigate.
We focus on the learning experience as a whole, by providing a set of modules, teaching plans, and other guides, as well as a curated role-play experience.
We have two targets:
For instructors, we want to create a βclass-in-a-boxβ that allows people to offer a modern and engaging learning experience. Currently, this is feasible at the advanced undergraduate and graduate level, though in the future we would like to provide a platform that can be adapted to a range of different grade and experience levels.
For self-guided learners, we want to create a βself-learning experienceβ that allows students to go from having zero knowledge of robotics to a graduate-level understanding.
In addition, the Duckietown platform is also suitable for research.
The video in Figure 1.1 is the βDuckumentaryβ, a documentary about the first version of the class, during Spring 2016.
Cannot open URL http://vimeo.com/api/v2/video/231688769.json | Traceback (most recent call last): | File "/home/duckietown/scm/duckuments/mcdp/src/mcdp_docs/videos.py", line 110, in get_vimeo_info | response = urllib2.urlopen(url) | File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen | return opener.open(url, data, timeout) | File "/usr/lib/python2.7/urllib2.py", line 435, in open | response = meth(req, response) | File "/usr/lib/python2.7/urllib2.py", line 548, in http_response | 'http', request, response, code, msg, hdrs) | File "/usr/lib/python2.7/urllib2.py", line 473, in error | return self._call_chain(*args) | File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain | result = func(*args) | File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default | raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) | HTTPError: HTTP Error 503: Service Unavailable
The video in Figure 1.2 is a documentary created by Red Hat on the current developments in self-driving cars.
If youβd like to know more about the educational experience, [1] present a more formal description of the course design for Duckietown: learning objectives, teaching methods, etc.
The video in Figure 1.3 shows some of the functionality of the platform.
If you would like to know more, the paper [2] describes the Duckiebot and its software. (With 30 authors, we made the record for a robotics conference!)
The original Duckietown class was at MIT in 2016 (Figure 2.1).
Duckietown was built by elves (Figure 2.2).
These are some advertisement videos we used.
Later that year, the Duckietown platform was also used in these classes:
In 2017, these four courses will be taught together, with the students interacting among institutions:
Furthermore, the Duckietown platform is used also in the following universities:
to write
DuckietownHS is inspired by the Duckietown project and targeted for high schools. The goal is to build and program duckiebots capable of moving autonomously on the streets of Duckietown. The technical objectives of DuckietownHS are simplified compared to those of the Duckietown project intended for universities so it is perfectly suited to the technical knowledge of the classes involved. The purpose is to create self-driving DuckiebotHS vehicles which can make choices and move autonomously on the streets of Duckietown, using sensors installed on the vehicles and special road signs positioned within Duckietown.
Once DuckiebotHS have been assembled and programmed to meet the specifications contained in this document and issued by the βcustomerβ Perlatecnica, special missions and games will be offered for DuckiebotHS. The participants can also submit their own missions and games.
Just like the university project, DuckietownHS is an open source project, a role-playing game, a means to raise awareness on the subject and a learning experience for everyone involved. The project is promoted by the non-profit organization Perlatecnica based in Italy.
The project has two main purposes:
Perlatecnica assumes the role of the customer and commissions the Duckietown Engineering company to design and construct the Duckietown and DuckiebotHS. It will provide all necessary product requirements and will assume the responsibility to validate the compliancy of all deliverables to the required specifications.
The project consists in the design and realization of DuckiebotHS and DuckietownHS. They must have the same characteristics as the city of the University project as far as the size and color of the delimiting roadway bands is concerned but with a different type of management of the traffic lights system that regulates the passage of DuckiebotHS at intersections. The DuckietownHS (DTHS) and DuckiebotHS (DBHS) are defined in the documentation and there is little room for the DTE to make its own choices in terms of design. The reason for this is that the DBHS produced by the different DTEβs need to be identical from a hardware point of view so that the software development makes the difference.
The purchase of the necessary materials is the first step to take. For both DTHS and DBHS a list of these materials is provided with links to possible sellers. Even though Amazon is typically indicated as a seller this is nothing more than an indication to facilitate the purchase for those less experienced. It is left to the individual DTE to choose where to buy the required parts. It is allowed to buy and use parts that are not on the list but this is not recommended as they will make the Duckiebot unfit to enter in official competitions. When necessary an assembly tutorial will be provided together with the list of materials. Once the DTHS city and the DBHS robots have been assembled, the next step will be the development of the software for the running of both the city and the DuckiebotHS. The city and the Duckiebot run on a board based on a microcontroller STM32 from STMicroelectronics the Nucleo F401RE that will be programmed via the online development environment mbed. Perlatecnica will not release any of the official codes necessary for the navigation of the DuckiebotHS as these are owned by the DTE who developed them. The full standard document is available on the project official web site.
Each DTE may release the source code under a license Creative Commons CC BY-SA 4.0.
Once you have completed the assembly of all the parts that make up the Duckietown and DuckiebotHS you should start programming the microcontroller so that the Duckiebot can move independently.
The basic mission of the DuckiebotHS is to move autonomously on the roads respecting the road signs and traffic lights, choosing a random journey and without crashing into other DuckiebotHS.
For the development of the code, there are no architectural constraints, but we recommend proceeding with order and to focus primarily on its major functions and not on a specific mission.
The main functions are those of perception and movement.
Moving around in DuckietownHS, the DuckiebotHS will have to drive on straight roads, make 90 degree curves while crossing an intersection but also make other unexpected curves. While doing all this the Duckiebot can be supported by a gyroscope that provides guidance to the orientation of the vehicle.
Duckietown is an international effort. Many educational institutions have adopted the platform and used to teach robotics and related subjects. If you have used Duckietown in any of your course and cannot find a mention to it here, contact us.
add contact email
Here, we provide a chronologically ordered compendium of the Duckietown related learning experiences worldwide.
Duckietown was created in 2016.
Location: United States of America, Cambridge, Massachusetts
Course title: 2.166 Vehicle Autonomy
Instructors: plenty
Educational Level: Graduate
Time period:
Link:
Highlights: Role-playing experience (Duckietown Engineering Co.), public demo
Summary:
Location: Hsinchu, Taiwan
Course title: Robotic Vision
Instructors: Prof. Nick Wang
Educational Level: Graduate
Time period:
Link:
Summary:
Highlights:
Location:
Course title:
Instructors:
Educational Level:
Time period:
Link:
Summary:
Highlights:
Location:
Course title:
Instructors:
Educational Level:
Time period:
Link:
Summary:
Highlights:
Location: ZΓΌrich
Course title: Autonomous Mobility on Demand (AMOD): from car to fleet (151-0323-00L)
Instructors: E. Frazzoli, A. Censi
Educational Level: Graduate
Time period: Sept. - Dec. 2017
Link: Official Course Website
Summary: 46 students
Highlights:
Location: Montreal
Course title: IFT 6080 Sujets en exploitation des ordinateurs
Instructors: Liam Paull
Educational Level: Graduate
Time period: Sept. - Dec. 2017
Link: Official Course Website
Summary: 12 students admitted spanning across UniversitΓ© de MontrΓ©al, Γcole Polytechnic, McGill University and Concordia University
Highlights:
Location: Chicago
Course title: TTIC 31240 Self-driving Vehicles: Models and Algorithms for Autonomy
Instructors: Matthew R. Walter
Educational Level: Undergraduate/Graduate
Time period: Sept. - Dec. 2017
Link: Official Course Website
Summary: 12 students admitted from TTI-Chicago and the University of Chicago
Highlights:
Location: Hsinchu, Taiwan
Course title: Creative Software Project - Autonomous Vehicle
Instructors: Hsueh-Cheng βNickβ Wang
Educational Level: Undergraduate
Time period: Sept. 2017 - Jan. 2018
Link: Official Course Website
Summary: 57 students (and 10 teaching assistants) admitted from National Chiao Tung University
Highlights: Marine Robot, Multi-Robot Patrolling, Robot Terrian Discrimination in Gazebo, and Depth Lane Following
If you are an instructor, please jump to Section 4.2 - Duckietown for instructors.
If you are a self-guided learner, please jump to Section 4.3 - Duckietown for self-guided learners.
If you are a company, and interested in working with Duckietown, please jump to Section 4.4 - Introduction for companies.
to write
to write
to write
add link to Facebook
add link to Mailing list
add link to Slack?
If you want to contribute to the softwareβ¦
If you want to contribute to the hardwareβ¦
If you want to contribute to the documentationβ¦
If you want to contribute to the disseminationβ¦
What is Duckietown?
Duckietown is a low-cost educational and research platform.
Is Duckietown free to use?
Yes. All materials are released according to an open source license.
Is everything ready?
Not quite! Please sign up to our mailing list to get notified when things are a bit more ready.
How can I start?
See the section First Steps.
How can I help?
If you would like to help actively, please email duckietown@mit.edu.
I want to build my own Duckiebot. How do I get started?
to write
How large a class can it be? I teach large classes.
to write
What is the budget for the robot?
to write
I want to teach a Duckietown class. How do I get started?
Please get in touch with us at duckietown@mit.edu. We will be happy to get you started and sign you up to the Duckietown instructors mailing list.
Why the duckies?
Compared to other educational robotics projects, the presence of the duckies is what makes this project stand out. Why the duckies?
We want to present robotics in an accessible and friendly way.
copy usual discussion from somewhere else.
add picture of kids with Duckiebots.
All the documentation is in the repository duckietown/duckuments.
The documentation is written as a series of small files in Markdown format.
It is then processed by a series of scripts to create this output:
The simplest way to contribute to the documentation is to click any of the βββ icons next to the headers.
They link to the βeditβ page in Github. There, one can make and commit the edits in only a few seconds.
In the following, we are going to assume that the documentation system is
installed in ~/duckuments. However, it can be installed anywhere.
We are also going to assume that you have setup a Github account with working public keys.
We are also going to assume that you have installed the duckietown/software in ~/duckietown.
On Ubuntu 16.04, these are the dependencies to install:
$ sudo apt install libxml2-dev libxslt1-dev
$ sudo apt install libffi6 libffi-dev
$ sudo apt install python-dev python-numpy python-matplotlib
$ sudo apt install virtualenv
$ sudo apt install bibtex2html pdftk
$ sudo apt install imagemagick
duckuments repoDownload the duckietown/duckuments repository in the ~/duckuments directory:
$ git lfs clone --depth 100 git@github.com:duckietown/duckuments ~/duckuments
Here, note we are using git lfs clone β itβs much faster, because it downloads
the Git LFS files in parallel.
If it fails, it means that you do not have Git LFS installed. See Unit J-28 - Git LFS.
The command --depth 100 tells it we donβt care about the whole history.
Next, we will create a virtual environment using inside the ~/duckuments
directory. Make sure you are running Python 2.7.x. Python 3.x is not supported at the moment.
Change into that directory:
$ cd ~/duckuments
Create the virtual environment using virtualenv:
$ virtualenv --system-site-packages deploy
Other distributions: In other distributions you might need to use venv instead of virtualenv.
Activate the virtual environment:
$ source ~/duckuments/deploy/bin/activate
mcdp external repositoryMake sure you are in the directory:
$ cd ~/duckuments
Clone the mcdp external repository, with the branch duckuments.
$ git clone -b duckuments git@github.com:AndreaCensi/mcdp
Install it and its dependencies:
$ cd ~/duckuments/mcdp
$ python setup.py develop
If you get a permission error here, it means you have not properly activated the virtual environment.
Other distributions: If you are not on Ubuntu 16, depending on your system, you might need to install these other dependencies:
$ pip install numpy matplotlib
You also should run:
$ pip install -U SystemCmd==2.0.0
Make sure you have deployed and activated the virtual environment. You can check
this by checking which python is active:
$ which python
/home/user/duckuments/deploy/bin/python
To compile the master versions of the docs, run:
$ make master-clean master
To see the result, open the file
./duckuments-dist/master/duckiebook/index.html
If you want to do incremental compilation, you can omit the clean and just
use:
$ make master
This will be faster. However, sometimes it might get confused. At that point,
do make master-clean.
To compile the Fall 2017 versions of the docs, run:
$ make fall2017-clean fall2017
To see the result, open the file
./duckuments-dist/master/duckiebook/index.html
For incremental compilation, use:
$ make fall2017
There is also the option to compile one single file.
To do this, use:
$ ./compile-single path to .md file
This is the fastest way to see the results of the editing; however, there are limitations:
This is the basic workflow:
yourname-branch in the duckuments repository.yourname-branch branch.make master to make sure it compiles.yourname-branch branch.duckietown/gardeners.Create a pull request from the command-line using hub.
First, see the section Unit B-7 - Markduck troubleshooting for common problems and their resolution.
Please report problems with the duckuments using the duckuments issue tracker.
If it is urgent, please tag people (Andrea); otherwise these are processed in batch mode every few days.
If you have a problem with a generated PDF, please attach the offending PDF.
If you say something like βThis happens for Figure 3β, then it is hard to know which figure you are referencing exactly, because numbering changes from commit to commit.
If you want to refer to specific parts of the text, please commit all your work on your branch, and obtain the name of the commit using the following commands:
$ git -C ~/duckuments rev-parse HEAD # commit for duckuments
$ git -C ~/duckuments/mcdp rev-parse HEAD # commit for mcdp
The Duckiebook is written in Markduck, a Markdown dialect.
It supports many features that make it possible to create publication-worthy materials.
The Duckiebook is written in a Markdown dialect.
Use the syntax β![name]β for describing the variables in the code.
For example, to obtain:
$ ssh robot name.local
Use the following:
For example, to obtain:
$ ssh ![robot name].local
Make sure to quote (with 4 spaces) all command lines. Otherwise, the dollar symbol confuses the LaTeX interpreter.
Use the string
to write the dollar symbol
$
, otherwise it gets confused with LaTeX math materials. Also notice
that you should probably use βUSDβ to refer to U.S. dollars.$
Other symbols to escape are shown in Table 2.1.
use $ |
instead of $ |
use ` |
instead of ` |
use &#lt; |
instead of < |
use &#gt; |
instead of > |
Use the kbd element for keystrokes.
For example, to obtain:
Press a then Ctrl-C.
use the following:
Press <kbd>a</kbd> then <kbd>Ctrl</kbd>-<kbd>C</kbd>.
For any element, adding an attribute called figure-id
with value fig:figure ID or tab:table ID
will create a figure that wraps the element.
For example:
<div figure-id="fig:figure ID">
figure content
</div>
It will create HMTL of the form:
<div id='fig:code-wrap' class='generated-figure-wrap'>
<figure id='fig:figure ID' class='generated-figure'>
<div>
figure content
</div>
</figure>
</div>
To add a caption, add an attribute figure-caption:
<div figure-id="fig:figure ID" figure-caption="This is my caption">
figure content
</div>
Alternatively, you can put anywhere an element figcaption with ID figure id:caption:
<element figure-id="fig:figure ID">
figure content
</element>
<figcaption id='fig:figure ID:caption'>
This the caption figure.
</figcaption>
To refer to the figure, use an empty link:
Please see [](#fig:figure ID).
The code will put a reference to βFigure XXβ.
You can also create subfigures, using the following syntax.
<div figure-id="fig:big">
<figcaption>Caption of big figure</figcaption>
<div figure-id="subfig:first" figure-caption="Caption 1">
<p style='width:5em;height:5em;background-color:#eef'>first subfig</p>
</div>
<div figure-id="subfig:second" figure-caption="Caption 2">
<p style='width:5em;height:5em;background-color:#fee'>second subfig</p>
</div>
</div>
This is the result:
first subfig
second subfig
By default, the subfigures are displayed one per line.
To make them flow horizontally, add figure-class="flow-subfigures" to the external figure div. Example:
first subfig
second subfig
The shortcuts col2, col3, col4, col5
are expanded in tables with 2, 3, 4 or 5 columns.
The following code:
<col2 figure-id="tab:mytable" figure-caption="My table">
<span>A</span>
<span>B</span>
<span>C</span>
<span>D</span>
</col2>
gives the following result:
| A | B |
| C | D |
labels-row1 and labels-row1Use the classes labels-row1 and labels-row1 to make pretty tables like the following.
labels-row1: the first row is the headers.
labels-col1: the first column is the headers.
class="labels-col1"| header A | B | C | 1 |
| header D | E | F | 2 |
| header G | H | I | 3 |
class="labels-row1"| header A | header B | header C |
| D | E | F |
| G | H | I |
| 1 | 2 | 3 |
You give IDs to headers using the format:
### header title {#topic ID}
For example, for this subsection, we have used:
### Establishing names of headers {#establishing}
With this, we have given this header the ID βestablishingβ.
Some time ago, if there was a section called
## My section
then it would be assigned the ID βmy-sectionβ.
This behavior has been removed, for several reasons.
One is that if you donβt see the ID then you will be tempted to just change the name:
## My better section
and silently the ID will be changed to βmy-better-sectionβ and all the previous links will be invalidated.
The current behavior is to generate an ugly link like βautoid-209u31jβ.
This will make it clear that you cannot link using the PURL if you donβt assign an ID.
Also, I would like to clarify that all IDs are global (so itβs easy to link stuff, without thinking about namespaces, etc.).
Therefore, please choose descriptive IDs, with at least two IDs.
E.g. if you make a section called
## Localization {#localization}
thatβs certainly a no-no, because βlocalizationβ is too generic.
## Localization {#intro-localization}
Also note that you donβt need to add IDs to everything, only the things that people could link to. (e.g. not subsubsections)
You can use the syntax:
[](#topic ID)
to refer to the header.
You can also use some slightly more complex syntax that also allows to link to only the name, only the number or both (Table 2.5).
|
| See Subsection 2.8.1 - Establishing names of headers |
|
| See Establishing names of headers. |
|
| See 2.8.1. |
|
| See Subsection 2.8.1 - Establishing names of headers. |
You are encouraged to put links to the documentation from the code or scripts.
To do so, use links of the form:
http://purl.org/dth/topic ID
Here βdthβ stands for βDuckietown Helpβ. This link will get redirected to
the corresponding document on the website.
For example, you might have a script whose output is:
$ rosrun mypackage myscript
Error. I cannot find the scuderia file.
See: http://purl.org/dth/scuderia
When the user clicks on the link, they will be redirected to Section 4.2 - The βscuderiaβ (vehicle database).
The system supports parsing of some special paragraphs.
some of these might be redundant and will be eliminated. For now, I am documenting what is implemented.
A special paragraph is marked by a special prefix. The list of special prefixes is given in the next section.
There must be an empty line before a special paragraph; this is because in Markdown a paragraph starts only after an empty line.
This is checked automatically, and the compilation will abort if the mistake is found.
For example, this is invalid:
See: this book
See: this other book
This is correct:
See: this book
See: this other book
Similarly, this is invalid:
Author: author
Maintainer: maintainer
and this is correct:
Author: author
Maintainer: maintainer
TODO: todo
todo
TOWRITE: towrite
towrite
Task: task
task
Assigned: assigned
assigned
Remark: remark
remark
Note: note
note
Warning: warning
warning
Symptom: symptom
symptom
Resolution: resolution
resolution
Bad: bad
bad
Better: better
better
Q: question
question
A: answer
answer
Maintainer: maintainer
maintainer
Author: author
Point of Contact: Point of Contact name
Point of Contact name
Slack channel: slack channel name
slack channel name
See: see
see
Reference: reference
reference
Requires: requires
requires
Results: results
results
Next steps: next steps
Recommended: recommended
recommended
See also: see also
see also
div environmentsFor these, note the rules:
markdown="1".div and before the closing /div.<div class='example-usage' markdown="1">
This is how you can use `rosbag`:
$ rosbag play log.bag
</div>
This is how you can use rosbag:
$ rosbag play log.bag
<div class='check' markdown="1">
Check that you didn't forget anything.
</div>
Check that you didnβt forget anything.
<div class='requirements' markdown="1">
List of requirements at the beginning of setup chapter.
</div>
List of requirements at the beginning of setup chapter.
There are three environments: βcommentβ, βquestionβ, βdoubtβ, that result in boxes that can be expanded by the user.
These are the one-paragraph forms:
Comment: this is a comment on one paragraph.
Question: this is a question on one paragraph.
this is a question on one paragraph.
Doubt: I have my doubts on one paragraph.
I have my doubts on one paragraph.
These are the multiple-paragraphs forms:
<div class='comment' markdown='1'>
A comment...
A second paragraph...
</div>
A commentβ¦
A second paragraphβ¦
<div class='question' markdown='1'>
A question...
A second paragraph...
</div>
A questionβ¦
A second paragraphβ¦
<div class='doubt' markdown='1'>
A question...
Should it not be:
$ alternative command
A second paragraph...
</div>
A questionβ¦
Should it not be:
$ alternative command
A second paragraphβ¦
Working knowledge of LaTeX.
You can use $\LaTeX$ math, environment, and references. For example, take a look at
$$ x^2 = \int_0^t f(\tau)\ \text{d}\tau $$
or refer to Proposition 1 - Proposition example.
Proposition example This is an example proposition: $2x = x + x$.
The above was written as in Listing 4.1.
You can use $\LaTeX$ math, environment, and references.
For example, take a look at
\[
x^2 = \int_0^t f(\tau)\ \text{d}\tau
\]
or refer to [](#prop:example).
\begin{proposition}[Proposition example]\label{prop:example}
This is an example proposition: $2x = x + x$.
\end{proposition}
For the LaTeX environments to work properly you must add a \label
declaration inside. Moreover, the label must have a prefix that is adequate
to the environment. For example, for a proposition, you must insert \label{prop:name}
inside.
The following table shows the list of the LaTeX environments supported and the label prefix that they need.
definition |
def:name |
proposition |
prop:name |
remark |
rem:name |
problem |
prob:name |
theorem |
thm:name |
lemma |
lem:name |
Examples of all environments follow.
\begin{definition} \label{def:lorem}
Lorem
\end{definition}
Lorem
\begin{proposition} \label{prop:lorem}
Lorem
\end{proposition}
Lorem
\begin{remark} \label{rem:lorem}
Lorem
\end{remark}
Lorem
\begin{problem} \label{prob:lorem}
Lorem
\end{problem}
Lorem
\begin{example} \label{exa:lorem}
Lorem
\end{example}
Lorem
\begin{theorem} \label{thm:lorem}
Lorem
\end{theorem}
Lorem
\begin{lemma} \label{lem:lorem}
Lorem
\end{lemma}
Lorem
I can also refer to all of them:
[](#def:lorem),
[](#prop:lorem),
[](#rem:lorem),
[](#prob:lorem),
[](#exa:lorem),
[](#thm:lorem),
[](#lem:lorem).
I can also refer to all of them: Definition 1, Proposition 2, Remark 1, Problem 1, Example 1, Theorem 1, Lemma 1.
We can refer to equations, such as \eqref{eq:one}:
\begin{equation} 2a = a + a \label{eq:one}\tag{1} \end{equation}
This uses align and contains \eqref{eq:two} and \eqref{eq:three}.
\begin{align} a &= b \label{eq:two}\tag{2} \\ &= c \label{eq:three}\tag{3} \end{align}
We can refer to equations, such as \eqref{eq:one}:
\begin{equation}
2a = a + a \label{eq:one}
\end{equation}
This uses `align` and contains \eqref{eq:two} and \eqref{eq:three}.
\begin{align}
a &= b \label{eq:two} \\
&= c \label{eq:three}
\end{align}
Note that referring to the equations is done using the syntax \eqref{eq:name},
rather than [](#eq:name).
The LaTeX symbols definitions are in a file called docs/symbols.tex.
Put all definitions there; if they are centralized it is easier to check that they are coherent.
You need to have installed bibtex2html.
The system supports Bibtex files.
Place *.bib files anywhere in the directory.
Then you can refer to them using the syntax:
[](#bib:bibtex ID)
For example:
Please see [](#bib:siciliano07handbook).
Will result in:
Please see [3].
In order to compile the figures into PDFs you need to have Inkscape installed. Instructions to download and install Inkscape are here.
To embed latex in your figures, you can add it directly to a file and save it as filename.svg file and save anywhere in the /docs directory.
You can run:
$ make process-svg-figs
And the SVG file will be compiled into a PDF figure with the LaTeX commands properly interpreted.
You can then include the PDF file in a normal way (Section 2.5 - Figures) using filename.pdf as the filename in the <img> tag.
It can take a bit of work to get the positioning of the code to appear properly on the figure.
It is possible to embed Vimeo videos in the documentation.
Do not upload the videos to your personal Vimeo account; they must all be posted to the Duckietown Engineering account.
This is the syntax:
<dtvideo src="vimeo:vimeo ID"/>
For example, this code:
<div figure-id="fig:example-embed">
<figcaption>Cool Duckietown by night</figcaption>
<dtvideo src="vimeo:152825632"/>
</div>
produces this result:
Depending on the output media, the result will change:
move-here tagIf a file contains the tag move-here, the fragment pointed
by the src attribute is moved at the place of the tag.
This is used for autogenerated documentation.
Syntax:
# Node `node`
<move-here src='#package-node-autogenerated'/>
You can insert comments using the HTML syntax for comments:
any text between β<!ββ and ββ>β is ignored.
# My section
<!-- this text is ignored -->
Let's start by...
You can refer to files in the repository by using:
See [this file](github:org=org,repo=repo,path=path,branch=branch).
The available keys are:
org (required): organization name (e.g. duckietown);repo (required): repository name (e.g. Software);path (required): the filename. Can be just the file name or also include directories;branch (optional) the repository branch; defaults to master;For example, you can refer to the file pkg_name/src/subscriber_node.py by using the following syntax:
See [this file](github:org=duckietown,repo=Software,path=pkg_name/src/subscriber_node.py)
You can also refer to a particular line:
This is done using the following parameters:
from_text (optional): reference the first line containing the text;from_line (optional): reference the line by number;For example, you can refer to the line containing βInitializeβ
of pkg_name/src/subscriber_node.py by using the following syntax:
For example, you can refer to [the line containing "Initialize"][link2]
of `pkg_name/src/subscriber_node.py` by using the following syntax:
[link2]: github:org=duckietown,repo=Software,path=pkg_name/src/subscriber_node.py,from_text=Initialize
You can also reference a range of lines, using the parameters:
to_text (optional): references the final line, by text;to_line (optional): references the final line, by number.You cannot give from_text and from_line at the same time.
You cannot give a to_... without the from_....
For example, this link refers to a range of lines: click it to see how Github highlights the lines from βInitializeβ to βspinβ.
This is the source of the previous paragraph:
For example, [this link refers to a range of lines][interval]: click it to see how Github highlights the lines from "Initialize" to "spin".
[interval]: github:org=duckietown,repo=Software,path=pkg_name/src/subscriber_node.py,from_text=Initialize,to_text=spin
In addition to referencing the files, you can also copy the contents of a file inside the documentation.
This is done by using the tag display-file.
For example, you can put a copy of pkg_name/src/subscriber_node.py using:
<display-file src="
github:org=duckietown,
repo=Software,
path=pkg_name/src/subscriber_node.py
"/>
and the result is the following automatically generated listing:
#!/usr/bin/env python
import rospy
# Imports message type
from std_msgs.msg import String
# Define callback function
def callback(msg):
s = "I heard: %s" % (msg.data)
rospy.loginfo(s)
# Initialize the node with rospy
rospy.init_node('subscriber_node', anonymous=False)
# Create subscriber
subscriber = rospy.Subscriber("topic", String, callback)
# Runs continuously until interrupted
rospy.spin()
subscriber_node.pyIf you use the from_text and to_text (or from_line and to_line), you can actually display part of a file. For example:
<display-file src="
github:org=duckietown,
repo=Software,
path=pkg_name/src/subscriber_node.py,
from_text=Initialize,
to_text=spin
"/>
creates the following automatically generated listing:
# Initialize the node with rospy
rospy.init_node('subscriber_node', anonymous=False)
# Create subscriber
subscriber = rospy.Subscriber("topic", String, callback)
# Runs continuously until interrupted
rospy.spin()
subscriber_node.pyThis part describes how to compile the PDF version.
The dependencies below are harder to install. If you donβt manage
to do it, then you only lose the ability to compile the PDF. You can do make compile
to compile the HTML version, but you cannot do make compile-pdf.
nodejsEnsure the latest version (>6) of nodejs is installed.
Run:
$ nodejs --version
6.xx
If the version is 4 or less, remove nodejs:
$ sudo apt remove nodejs
Install nodejs using the instructions at this page.
Next, install the necessary Javascript libraries using npm:
$ cd $DUCKUMENTS
$ npm install MathJax-node jsdom@9.3 less
nodejs installation problemsThe only pain point in the installation procedure has been the installation of nodejs packages using npm. For some reason, they cannot be installed globally (npm install -g).
Do not use sudo for installation. It will cause problems.
If you use sudo, you probably have to delete a bunch of directories,
such as: ~/duckuments/node_modules, ~/.npm, and ~/.node_modules, if they exist.
Install PrinceXML from this page.
Copy the ~/duckuments/fonts directory in ~/.fonts:
$ mkdir -p ~/.fonts # create if not exists
$ cp -R ~/duckuments/fonts ~/.fonts
and then rebuild the font cache using:
$ fc-cache -fv
To compile the PDF, use:
$ make compile-pdf
This creates the file:
./duckuments-dist/master/duckiebook.pdf
For these issues, see Unit B-8 - The Duckuments bot.
βInvalid XMLβ
βMarkdownβ doesnβt mean that you can put anything in a file. Except
for the code blocks, it must be valid XML. For example, if you use
β>β and β<β without quoting, it will likely
cause a compile error.
βTabs are evilβ
Do not use tab characters. The error message in this case is quite helpful in telling you exactly where the tabs are.
The error message contains ValueError: Suspicious math fragment 'KEYMATHS000ENDKEY'
You probably have forgotten to indent a command line by at least 4 spaces. The dollar in the command line is now being confused for a math formula.
Here are some common mistakes encountered.
There must be an empty line before the list starts.
This is correct:
I want to learn:
- robotics
- computer vision
- underwater basket weaving
This is incorrect:
I want to learn:
- robotics
- computer vision
- underwater basket weaving
and it will be rendered as follows:
I want to learn: - robotics - computer vision - underwater basket weaving
This is an advanced section mainly for Liam.
The book is published to a different repository called duckuments-dist and from there
published as book.duckietown.org.
There is a bot, called frankfurt.co-design.science, which is an AWS machine (somewhere in Frankfurt).
Every minute, it tries to do the following:
mcdp;duckuments;duckuments-dist.This process takes 4 minutes for an incremental change, and about 10-15 minutes for a big change, such as a change in headers IDs, which implies re-checking all cross-references.
There are logs you can access to see whatβs going on.
The high-level compilation log tells you in what phase of the cycle the bot is. Scroll to the bottom.
Ideally what you want to see is something like the following:
Starting
Mon Sep 11 10:49:04 CEST 2017
succeded html
succeded fall 2017
succeded upload
succeded split
succeded html upload
succeded PDF
succeded PDF upload
Mon Sep 11 10:54:21 CEST 2017
Done.
This shows that the compilation took 5 minutes.
Every two hours you will see something like this:
automatic-compile-cleanup killing everything
and the next iteration will take longer because it starts from scratch.
The last log is a live version of the compilation log. This might not be tremendously informative because it is very verbose.
Sometimes, itβs Github Pages that lags behind.
To check this, the bot also makes available the compilation output as a website
called book2.duckietown.org. You can take any URL starting with
book.duckietown.org, put book2, and you will see what is on the server.
This can identify if the problem is Github.
This chapter describes the conventions for writing the technical documentation.
The following holds for all technical writing.
The documentation is written in correct English.
Do not say βshouldβ when you mean βmustβ. βMustβ and βshouldβ have precise meanings and they are not interchangeable. These meanings are explained in this document.
βPleaseβ is unnecessary in technical documentation.
βPlease remove the SD card.β
βRemove the SD cardβ.
βThe pwd is
ubuntu.ββThe password is
ubuntu.ββTo create a ROS pkgβ¦β
βTo create a ROS packageβ¦β
βIf you are using pythonβ¦β
βIf you are using Pythonβ¦β
Do not use emojis.
Do not use ALL CAPS.
Make infrequent use of bold statements.
Do not use exclamation points.
behaviour
behavior
localisation
localization
Itβs ok to use βitβsβ instead of βit isβ, βcanβtβ instead of βcannotβ, etc.
All the filenames and commands must be enclosed in code blocks using Markdown backticks.
βEdit the ~/.ssh/config file using vi.β
βEdit the
~/.ssh/configfile usingvi.β
ssh etc. are not verbs.βCtrl-C from the command lineβ.
βUse Ctrl-C from the command lineβ.
Use either βlaptopβ or βduckiebotβ (not capitalized, as a hostname) as the prefix for the command line.
For example, for a command that is supposed to run on the laptop, use:
laptop $ cd ~/duckietown
It will become:
laptop $ cd ~/duckietown
For a command that must run on the Duckiebot, use:
duckiebot $ cd ~/duckietown
It will become:
duckiebot $ cd ~/duckietown
If the command is supposed to be run on both, omit the hostname:
$ cd ~/duckietown
βDuckiebotβ is always capitalized.
Use βRaspberry Piβ, not βPIβ, βraspiβ, etc.
These are other words frequently misspelled:
5 GHz WiFi
When the user must edit a file, just say: βedit /this/fileβ.
Writing down the command line for editing, like the following:
$ vi /this/file
is too much detail.
(If people need to be told how to edit a file, Duckietown is too advanced for them.)
Write the documentation as if every step succeeds.
Then, at the end, make a βTroubleshootingβ section.
Organize the troubleshooting section as a list of symptom/resolution.
The following is an example of a troubleshooting section.
This strange thing happens.
Maybe the camera is not inserted correctly. Remove and reconnect.
This other strange thing happens.
Maybe the plumbus is not working correctly. Try reformatting the plumbus.
Jacopo
This chapter is a draft.
We do not refer to teachers and students
We refer to learners. Instructors are learners ahead in the learning curve in comparison to other learners
We donβt examinate
We assess competences
A relevant contribution of modern educational theory is recognizing the importance of feedback in the learning process ().
We love feedback, as it stands at the foundation of control systems theory.
Here, we provide definitions of the key elements in a learning feedback loop, and analogies to their control systems counterparts.
Intended Learning Outcome
An intended learning outcome is a desired, measurable, output of the learning process.
Intended learning outcomes are:
the starting point in the construction of a learning activity ([1]),
more effective when formulated with active verbs,
the equivalent of reference trajectories, or setpoints, in control systems. They represent the ideal output of the controlled system.
Students will understand robotics
Students each build a Duckiebot, implement software and produce demos
Learning Activities
Methods chosen by the instructor to ease the learners achievement of the intended learning outcomes.
Active learning practices ([1]) have been shown to improve learning.
Instructor explains at the blackboard
Learners work in groups to achieve objectives
Learning activities are analogous to the the output of the controller (instructor), or input to the system (learners). They have to meet the requirements imposed by external constraints, not shown in
Assessment
An assessment is a procedure to quantify the level of fulfillment of learning outcomes.
An assessment is analogous to a sensor in a control system loop. It measures the systemβs output.
Examples of assessments include: quizzes, colloquia, produced documentation, homework, etc.
Here, we provide definitions for knowledge, skills and competences, in addition to describing their relationships (Figure 10.2).
Knowledge
Theoretical facts and information aimed at enabling understanding, and generating or improving skills.
Bayesian inference is handy piece of knowledge when doing robotics.
Practice
Practical procedures aimed at generating or improving skills, either directly or indirectly by improving knowledge.
Exercises and proofs can be used to practice different skills.
Skills
A proficiency, facility or dexterity that enables carrying out a function. Skills stem from knowledge, practice and/or aptitude. Skills can be clustered in cognitive, technical and interpersonal, respectively relating to ideas, things and people.
Analyzing tradeoffs between performances and constraints is a critical cognitive skill for robotics.
Python language is a useful technical skill in robotics.
Public speaking is a valuable interpersonal skill useful beyond robotics.
In Duckietown we formalize didactic indivisible units, or atoms, aimed at improving skills through knowledge and practice. Knowledge atoms are listed in XXX. We define as practice atoms:
add general reference to all learning atmos, folder atoms_30_learning_material
Exercise
An exercise is a practice atom aimed at improving technical skills. Exercises are listed in XXX.
Exercises are targeted to different βthingsβ to which technical skills are related. They may be mathematical exercises aimed at practicing a method, or they may be coding exercises aimed at practicing resolutions of hardware implementation challenges.
Proof
A proof is a practice atom aimed at improving cognitive skills.
Deriving the Kalman filter equations helps practice the idea that there is no better approach to state estimation for linear time invariant systems, with βwell behavedβ measurement and process noises.
Competences
Set of skills and/or knowledge that leads to superior performance in carrying out a function. Competences must be measurable.
Competences are desirable intended learning outcomes, and typically address the how of the learning process.
Programming is a competence. It requires a skill, e.g., Python, and knowledge, e.g., Bayesian inference, to know what to code. Practice can help improve knowledge or hone skills.
This chapter describes something that is not implemented yet.
Atom An atom is a concrete resource (text, video) that is the smallest unit that is individually addressable. It is indivisible.
Each atom as a type, as follows:
text
text/theory
text/setup
text/demo
text/exercise
text/reference
text/instructor-guide
text/quiz
video
video/lecture
video/instructable
video/screencast
video/demo
Atoms form a directed graph, called βsemantic graphβ.
Each node is an atom.
The graph has four different types of edges:
A βmoduleβ is an abstraction from the point of view of the teacher.
Module A module is a directed graph, where the nodes are either atoms or other modules, and the edges can be of the four types described in Subsection 11.1.2 - Semantic graph of atoms.
Because modules can contain other modules, they allow to describe hierarchical contents. For example, a class module is a module that contains other modules; a βdegreeβ is a module that contains βclassβ modules, etc.
Modules can overlap. For example, a βBasic Object Detectionβ and an βAdvanced Object Detectionβ module might have a few atoms in common.
Each atom has the following properties:
- and β_β). The ID is used for cross-referencing.
It is the same in all languages.There might be different versions of each atom. This is used primarily for dealing with translations of texts, different representations of the same image, Powerpoint vs Keynote, etc.
A version is a tuple of attributes.
The attributes are:
Language: A language code, such as en-US (default), zh-CN, etc.
Mime type: a MIME type.
Each atom version has:
Each document has a status value.
The allowed values are described in Table 11.1.
draft |
We just started working on it, and it is not ready for public consumption. |
beta |
Early reviewers should look at it now. |
ready |
The document is ready for everybody. |
recently-updated |
The document has been recently updated (less than 1 week) |
to-update |
A new pass is needed on this document, because it is not up to date anymore. |
deprecated |
The document is ready for everybody. |
For the text-like resources, they are described in Markdown files.
The name of the file does not matter.
All files are encoded in UTF-8.
Each file starts with a H1 header. The contents is the title.
The header has the following attributes:
{#ID})status, which should be value of the values in Table 11.1.lang ({lang=en-US}).type ({type=demo}).Here is an example of a header with all the attributes:
# Odometry calibration {#odometry-calibration lang=en-US type='text/theory' status=ready}
This first paragraph will be used as the "summary" for this text.
calibration.en.mdAnd this is how the Italian translation would look like:
# Calibrazione dell'odometria {#odometry-calibration lang=it type='text/theory' status=draft}
Questo paragrafo sarΓ usato come un sommario del testo.
calibration.it.mdIn the text, you describe the semantic graph using tags and IDs.
In Markdown, you can give IDs to sections using the syntax:
# Setup step 1 {#setup-step1}
This is the first setup step.
Then, when you write the second step, you can add a a semantic edge using the following.
# Setup step 2 {#setup-step2}
This is the second setup step.
Requires: You have completed the first step in [](#setup-step1).
The following table describes the syntax for the different types of semantic links:
| Requires | |
| Recommended | |
| Reference | |
| See also | |
Define a micro-format for this.
This part is not implemented yet.
Translations are organized file-by-file.
For every file name.md, name the translated file name.language code.md,
where the language code is one of the standard codes, and put it in the same directory.
For example, these could be a set of files, including a Chinese (simplified), italian, and Spanish translation:
representations.md
representations.zh-CN.md
representations.it.md
representations.es.md
The reason is that in this way you can check automatically from Git whether representations.zh-CN.md is up to date or representations.md has been modified since.
Here are some considerations for the writers of the original version, to make the translatorsβ job easier.
It is better to keep files smallish so that (1) the translation tasks can feel approachable by translators; (2) it is easier for the system to reason about the files.
Name all the headers with short, easy identifiers, and never change them.
All files are assumed to be encoded in UTF-8.
The header IDs should not be translated and should remain exactly the same. This will allow keeping track of the different translations.
For example, if this is the original version:
# Robot uprising {#robot-uprising}
Hopefully it will never happen.
Then the translated version should be:
# La rivolta dei robot {#robot-uprising}
Speriamo che non succeda.
In this section you will find information to obtain the necessary equipment for Duckietowns and different Duckiebot configurations.
nothing
Knowledge of Duckiebot configuration naming conventions, their components and functionalities.
We define different Duckiebot configurations depending on their time of use and hardware components. This is a good starting point if you are wondering what parts you should obtain to get started.
The configurations are defined with a root: DB17-, indicating the βbare bonesβ Duckiebot used in the Fall 2017 synchronized course, and an appendix y which can be the union (in any order) of any or all of the elements of the optional hardware set $\aset{O} = \{$w, j, d, p, l, c$\}$.
The elements of $\aset{O}$ are labels identifying optional hardware that aids in the development phase and enables the Duckiebot to talk to other Duckiebots. The labels stand for:
w: 5 GHz wireless adapter to facilitate streaming of images;
j: wireless joypad that facilitates manual remote control;
d: USB drive for additional storage space;
c: a different castor wheel to replace the preexisting omni-directional wheel;
l: includes LEDs, LED hat, bumpers and the necessary mechanical bits to set the bumpers in place. Note that the installation of the bumpers induces the replacement of a few DB17 components;
During the Fall 2017 course, three Duckietown Engineering Co. branches (Zurich, Montreal, Chicago) are using these configuration naming conventions. Moreover, all institutions release hardware to their Engineers in training in two phases. We summarize the configuration releases below.
DB17This is the minimal configuration for a Duckiebot. It is the configuration of choice for tight budgets or when operation of a single Duckiebot is more of interest than fleet behaviors.
Functions: A DB17 Duckiebot can navigate autonomously in a Duckietown, but cannot communicate with other Duckiebots.
Components: A βbare-bonesβ DB17 configuration includes:
| Chassis | USD 20 |
| Camera with 160-FOV Fisheye Lens | USD 22 |
| Camera Mount | USD 8.50 |
| 300mm Camera Cable | USD 2 |
| Raspberry Pi 3 - Model B | USD 35 |
| Heat Sinks | USD 5 |
| Power supply for Raspberry Pi | USD 7.50 |
| 16 GB Class 10 MicroSD Card | USD 10 |
| Mirco SD card reader | USD 6 |
| DC Motor HAT | USD 22.50 |
| Spliced USB-A power cable | USD 0 |
| 2 Stacking Headers | USD 2.50/piece |
| Battery | USD 20 |
| 16 Nylon Standoffs (M2.5 12mm F 6mm M) | USD 0.05/piece |
| 4 Nylon Hex Nuts (M2.5) | USD 0.02/piece |
| 4 Nylon Screws (M2.5x10) | USD 0.05/piece |
| 2 Zip Ties (300x5mm) | USD 9 |
Total cost for DB17 configuration |
USD 173.6 |
Description of components: Unit C-2 - Acquiring the parts (DB17-jwd)
Assembly instructions: Without videos, with videos
DB17-wThis configuration is the same as DB17 with the addition of a 5 Ghz wireless adapter.
Functions: This configuration has the same functionality of DB17. In addition, it equips the Duckiebot with a secondary, faster, Wi-Fi connection, ideal for image streaming.
Components:
DB17 |
USD 173.6 |
| Wireless Adapter (5 GHz) | USD 20 |
Total cost for DB17-w configuration |
USD 193.6 |
Description of components: Unit C-2 - Acquiring the parts (DB17-jwd)
Assembly instructions: Without videos, with videos
DB17-jThis configuration is the same as DB17 with the addition of a 2.4 GHz wireless joypad.
Functions: This configuration has the same functionality of DB17. In addition, it equips the Duckiebot with manual remote control capabilities. It is particularly useful for getting the Duckiebot our of tight spots or letting younger ones have a drive, in addition to providing handy shortcuts to different functions in development phase.
Components:
Description of components: Unit C-2 - Acquiring the parts (DB17-jwd)
Assembly instructions: Without videos, with videos
DB17-dThis configuration is the same as DB17 with the addition of a USB flash hard drive.
Functions: This configuration has the same functionality of DB17. In addition, it equips the Duckiebot with an external hard drive that is convenient for storing videos (logs) as it provides both extra capacity and faster data transfer rates than the microSD card in the Raspberry Pi. Moreover, it is easy to unplug it from the Duckiebot at the end of the day and bring it over to a computer for downloading and analyzing stored data.
Components:
DB17 |
USD 173.6 |
| Tiny 32GB USB Flash Drive | USD 12.50 |
Total cost for DB17-d configuration |
USD 186.1 |
Description of components: Unit C-2 - Acquiring the parts (DB17-jwd)
Assembly instructions: Without videos, with videos
DB17-cIn this configuration, the DB17 omni-directional wheel is replaced with a caster wheel.
Functions: The caster wheel upgrade provides a smoother ride.
Components:
DB17 |
USD 173.6 |
Caster (DB17-c) |
USD 6.55/4 pieces |
| 4 Standoffs (M3 12mm F-F) | USD 0.63/piece |
| 8 Screws (M3x8mm) | USD 4.58/100 pieces |
| 8 Split washer lock | USD 1.59/100 pieces |
Total cost for DB17-c configuration |
USD 178.25 |
update links of mechanical bits from M3.5 to M3.
The omni-directional caster wheel is included in the chassis package, so replacing it does not reduce the DB17 cost.
Description of components: Unit E-1 - Acquiring the parts (DB17-lc)
Assembly instructions: Unit E-3 - Assembling the Duckiebot (DB17-lc)
DB17-lIn this configuration the Duckiebot in equipped with the necessary hardware for controlling and placing 5 RGB LEDs on the Duckiebot. Differently from previous configurations that add or replace a single component, DB17-l introduces several hardware components that are all necessary for a proper use of the LEDs.
It may be convenient at times to refer to hybrid configurations including any of the DB17-jwcd in conjunction with a subset of the DB17-l components. In order to disambiguate, let the partial upgrades be defined as:
DB17-l1: adds a PWM hat to DB17, in addition to a short USB angled power cable and a M-M power wire;DB17-l2: adds a bumpers set to DB17, in addition to the mechanical bits to assemble it;DB17-l3: adds a LED hat and 5 RGB LEDs to DB17-l1l2, in addition to the F-F wires to connect the LEDs to the LED board.introducing the PWM hat in DB17-l1 induces a replacement of the spliced cable powering solution for the DC motor hat. Details can be found in Unit E-3 - Assembling the Duckiebot (DB17-lc).
Functions: DB17-l is the necessary configuration to enable communication between Duckiebots, hence fleet behaviors (e.g., negotiating the crossing of an intersection). Subset configurations are sometimes used in a standalone way for: (DB17-l1) avoid using a sliced power cable to power the DC motor hat in DB17, and (DB17-l2) for purely aesthetic reasons.
Components:
DB17 |
USD 173.6 |
PWM/Servo HAT (DB17-l1) |
USD 17.50 |
Power Cable (DB17-l1) |
USD 7.80 |
Male-Male Jumper Wire (150mm) (DB17-l1) |
|
| USD 1.95 | Bumper set (DB17-l2) |
| USD 7 (custom made) | 8 M3x10 pan head screws (DB17-l2) |
| USD 7 (custom made) | 8 M3 nuts (DB17-l2) |
| USD 7 (custom made) | Bumpers (DB17-l2) |
| USD 7 (custom made) | |
LEDs (DB17-l3) |
USD 10 |
LED HAT (DB17-l3) |
USD 28.20 for 3 pieces |
20 Female-Female Jumper Wires (300mm) (DB17-l3) |
USD 8 |
4 4 pin female header (DB17-l3) |
USD 0.60/piece |
12 pin male header (DB17-l3) |
USD 0.48/piece |
2 16 pin male
header (DB17-l3) |
USD 0.61/piece |
3 pin male header (DB17-l3) |
USD 0.10/piece |
2 pin female shunt jumper (DB17-l3) |
USD 2/piece |
40 pin female header (DB17-l3) |
USD 1.50 |
5 200 Ohm resistors (DB17-l3) |
USD 0.10/piece |
10 130 Ohm resistors (DB17-l3) |
USD 0.10/piece |
Total for DB17-l configuration |
USD 305 |
Description of components: Unit E-1 - Acquiring the parts (DB17-lc)
Assembly instructions: Unit E-3 - Assembling the Duckiebot (DB17-lc)
All branches release their hardware in two phases, namely a and b.
First release (DB17-Zurich-a): is a DB17-wjd.
Second release (DB17-Zurich-b): is a DB17-wjdcl.
First release (DB17-Montreal-a): is a hybrid DB17-wjd + PWM hat (or DB17-wjdl1).
Second release (DB17-Montreal-b): is a DB17-wjdl.
The Montreal branch is not implementing the DB17-c configuration.
First release (DB17-Chicago-a): is a DB17-wjd.
Second release (DB17-Chicago-b): is a DB17-wjdl.
The Chicago branch is not implementing the DB17-c configuration.
DB17-jwd)The trip begins with acquiring the parts. Here, we provide a link to all bits and pieces that are needed to build a Duckiebot, along with their price tag. If you are wondering what is the difference between different Duckiebot configurations, read this.
In general, keep in mind that:
Cost: USD 174 + Shipping Fees (minimal configuration DB17)
Time: 15 days (average shipping for cheapest choice of components)
A kit of parts ready to be assembled in a DB17 or DB17-wjd configuration.
| Chassis | USD 20 |
| Camera with 160-FOV Fisheye Lens | USD 22 |
| Camera Mount | USD 8.50 |
| 300mm Camera Cable | USD 2 |
| Raspberry Pi 3 - Model B | USD 35 |
| Heat Sinks | USD 5 |
| Power supply for Raspberry Pi | USD 7.50 |
| 16 GB Class 10 MicroSD Card | USD 10 |
| Mirco SD card reader | USD 6 |
| DC Motor HAT | USD 22.50 |
| 2 Stacking Headers | USD 2.50/piece |
| Battery | USD 20 |
| 16 Nylon Standoffs (M2.5 12mm F 6mm M) | USD 0.05/piece |
| 4 Nylon Hex Nuts (M2.5) | USD 0.02/piece |
| 4 Nylon Screws (M2.5x10) | USD 0.05/piece |
| 2 Zip Ties (300x5mm) | USD 9 |
Wireless Adapter (5 GHz) (DB17-w) |
USD 20 |
Joypad (DB17-j) |
USD 10.50 |
Tiny 32GB USB Flash Drive (DB17-d) |
USD 12.50 |
Total for DB17 configuration |
USD 173.6 |
Total for DB17-w configuration |
USD 193.6 |
Total for DB17-j configuration |
USD 184.1 |
Total for DB17-d configuration |
USD 186.1 |
Total for DB17-wjd configuration |
USD 216.6 |
We selected the Magician Chassis as the basic chassis for the robot (Figure 2.1).
We chose it because it has a double-decker configuration, and so we can put the battery in the lower part.
The chassis pack includes 2 DC motors and wheels as well as the structural part, in addition to a screwdriver and several necessary mechanical bits (standoffs, screws and nuts).
The Raspberry Pi is the central computer of the Duckiebot. Duckiebots use Model B (Figure 2.2) ( A1.2GHz 64-bit quad-core ARMv8 CPU, 1GB RAM), a small but powerful computer.
We want a hard-wired power source (5VDC, 2.4A, Micro USB) to supply the Raspberry Pi (Figure 2.3) while not driving. This charger can double down as battery charger as well.
Students in the ETHZ-Fall 2017 course will receive a converter for US to CH plug.
The Raspberry Pi will heat up significantly during use. It is warmly recommended to add heat sinks, as in Figure 2.4. Since we will be stacking HATs on top of the Raspberry Pi with 15 mm standoffs, the maximum height of the heat sinks should be well below 15 mm. The chip dimensions are 15x15mm and 10x10mm.
The MicroSD card (Figure 2.5) is the hard disk of the Raspberry Pi. 16 GB of capacity are sufficient for the system image.
A microSD card reader (Figure 2.6) is useful to copy the system image to a Duckiebot from a computer to the Raspberry Pi microSD card, when the computer does not have a native SD card slot.
The Camera is the main sensor of the Duckiebot. All versions equip a 5 Mega Pixels 1080p camera with wide field of view ($160^\circ$) fisheye lens (Figure 2.7).
The camera mount (Figure 2.8) serves to keep the camera looking forward at the right angle to the road (looking slightly down). The front cover is not essential.
The assembled camera (without camera cable), is shown in (Figure 2.9).
A longer (300 mm) camera cable Figure 2.10 makes assembling the Duckiebot easier, allowing for more freedom in the relative positioning of camera and computational stack.
We use the DC Stepper motor HAT (Figure 2.11) to control the DC motors that drive the wheels. This item will require soldering to be functional. This HAT has dedicate PWM and H-bridge for driving the motors.
We use a long 20x2 GPIO stacking header (Figure 2.12) to connect the Raspberry Pi with the DC Motor HAT. This item will require soldering to be functional.
The battery (Figure 2.13) provides power to the Duckiebot.
We choose this battery because it has a good combination of size (to fit in the lower deck of the Magician Chassis), high output amperage (2.4A and 2.1A at 5V DC) over two USB outputs, a good capacity (10400 mAh) at an affordable price. The battery linked in the table above comes with two USB to microUSB cables.
We use non electrically conductive standoffs (M2.5 12mm F 6mm M), nuts (M2.5), and screws (M2.5x10mm) to hold the Raspberry Pi to the chassis and the HATs stacked on top of the Raspberry Pi.
The Duckiebot requires 8 standoffs, 4 nuts and 4 screws.
Two 300x5mm zip ties are needed to keep the battery at the lower deck from moving around.
DB17-wThe Edimax AC1200 EW-7822ULC 5 GHz wireless adapter (Figure 2.16) boosts the connectivity of the Duckiebot, especially useful in busy Duckietowns (e.g., classroom). This additional network allows easy streaming of images.
DB17-jThe joypad is used to manually remote control the Duckiebot. Any 2.4 GHz wireless controller (with a tiny USB dongle) will do.
The model linked in the table (Figure 2.17) does not include batteries.
2 AA 1.5V batteries (Figure 2.18).
DB17-dIn configuration DB17-d, the Duckiebot is equipped with an βexternalβ hard drive (Figure 2.19). This add-on is very convenient to store logs during experiments and later port them to a workstation for analysis. It provides storage capacity and faster data transfer than the MicroSD card.
DB17)Shiying
Parts: Duckiebot DB17 parts. The acquisition process is explained in Unit C-2 - Acquiring the parts (DB17-jwd). The configurations are described in Unit C-1 - Duckiebot configurations. In particular:
Tools: Solderer
Experience: some novice-level experience with soldering.
Time: 30 minutes
Soldered DC Motor HAT
It is better to be safe than sorry. Soldering is a potentially hazardous activity. There is a fire hazard as well as the risk of inhaling toxic fumes. Stop a second and make sure you are addressing the safety standards for soldering when following these instructions. If you have never soldered before, seek advice.
There is a general rule in soldering: solder the components according to their height, from lowest to highest.
In this instruction set we will assume you have soldered something before and are acquainted with the soldering fundamentals. If not, before proceeding, read this great tutorial on soldering:
Take the GPIO stacking header Figure 3.1 out of Duckiebox and sort the following components from DC motor HAT package:
Adafruit DC/Stepper Motor HAT for Raspberry Pi
2-pin terminal block (2x), 3-pin terminal block (1x)
1) Make a 5 pin terminal block by sliding the included 2 pin and 3 pin terminal blocks into each other Figure 3.3.
2) Slide this 5 pin block through the holes just under βM1 GND M2β on the board. Solder it on (we only use two motors and do not need connect anything at the βM3 GND M4β location) (Figure 3.6);
3) Slide a 2 pin terminal block into the corner for power. Solder it on. (Figure 3.5);
4) Slide in the GPIO Stacking Header onto the 2x20 grid of holes on the edge opposite the terminal blocks and with vice versa direction (Figure 3.4). Solder it on.
stick the GPIO Stacking Header from bottom to top, different orientation than terminal blocks (from top to bottom).
DB17)In configuration DB17 we will need a cable to power the DC motor HAT from the battery. The keen observer might have noticed that such a cable was not included in the DB17 Duckiebot parts chapter. Here, we create this cable by splitting open any USB-A cable, identifying and stripping the power wires, and using them to power the DC motor HAT. If you are unsure about the definitions of the different Duckiebot configurations, read Unit C-1 - Duckiebot configurations.
It is important to note that these instructions are relevant only for assembling a DB17-wjdc configuration Duckiebot (or any subset of it). If you intend to build a DB17-l configuration Duckiebot, you can skip these instructions.
One male USB-A to anything cable.
A pair of scissors.
A multimeter (only if you are not purchasing the suggested components)
Time: 5 minutes
One male USB-A to wires power cable
The following video shows how to prepare the USB power cable for the configuration DB17.
To begin with, find a male USB-A to anything cable.
If you have purchased the suggested components listed in Unit C-2 - Acquiring the parts (DB17-jwd), you can use the longer USB cable contained inside the battery package (Figure 4.2), which will be used as an example in these instructions.
Put the shorter cable back in the box, and open the longer cable (Figure 4.3)
Make sure the USB cable is unplugged from any power source before proceeding.
Take the scissors and cut it (Figure 4.4) at the desired length from the USB-A port.
The cut will look like in Figure 4.5.
Paying attention not to get hurt, strip the external white plastic. A way to do so without damaging the wires is shown in Figure 4.6.
After removing the external plastic, you will see four wires: black, green, white and red (Figure 4.7).
Once the bottom part of the external cable is removed, you will have isolated the four wires (Figure 4.8).
Make sure the USB cable is unplugged from any power source before proceeding.
Once you have isolated the wires, strip them, and use the scissors to cut off the data wires (green and white, central positions) (Figure 4.9).
If you are not using the suggested cable, or want to verify which are the data and power wires, continue reading.
If you are using the USB-A cable from the suggested battery pack, black and red are the power wires and green and white are instead for data.
If you are using a different USB cable, or are curious to verify that black and red actually are the power cables, take a multimeter and continue reading.
Plug the USB port inside a power source, e.g., the Duckiebotβs battery. You can use some scotch tape to keep the cable from moving while probing the different pairs of wires with a multimeter. The voltage across the pair of power cables will be roughly twice the voltage between a power and data cable. The pair of data cables will have no voltage differential across them. If you are using the suggested Duckiebot battery as power source, you will measure around 5V across the power cables (Figure 4.10).
You are now ready to secure the power wires to the DC motor HAT power pins. To do so though, you need to have soldered the boards first. If you have not done so yet, read Unit C-3 - Soldering boards (DB17).
If you have soldered the boards already, you may test correct functionality of the newly crafted cable. Connect the battery with the DC motor HAT by making sure you plug the black wire in the pin labeled with a minus: - and the red wire to the plus: + (Figure 4.11).
DB17-jwd)Shiying Li
Once you have received the parts and soldered the necessary components, it is time to assemble them in a Duckiebot. Here, we provide the assembly instructions for configurations DB17-wjd.
Duckiebot DB17-wjd parts. The acquisition process is explained in Unit C-2 - Acquiring the parts (DB17-jwd).
Having soldered the DB17-wjd parts. The soldering process is explained in Unit C-3 - Soldering boards (DB17).
Having prepared the power cable. The power cable preparation is explained in Unit C-4 - Preparing the power cable (DB17). Note: Not necessary if you intend to build a DB17-l configuration.
Having installed the image on the MicroSD card. The instructions on how to reproduce the Duckiebot system image are in Unit C-7 - Reproducing the image.
Time: about 40 minutes.
An assembled Duckiebot in configuration DB17-wjd.
The FAQ section at the bottom of this page may already answer some of you comments, questions or doubts.
While assembling the Duckiebot, try to make as symmetric (along the longitudinal axis) as you can. It will help going forward.
Open the Magician chassis package (Figure 5.1) and take out the following components:
You wonβt need the battery holder and speed board holder (on the right side in Figure 5.1).
Insert the motor holders on the chassis-bottom and put the motors as shown in the figure below (with the longest screws (M3x30) and M3 nuts).
Orient the motors so that their wires are inwards, i.e., towards the center of the chassis-bottom. The black wires should be closer to the chassis-bottom to make wiring easier down the line.
if your Magician Chassis package has unsoldered motor wires, you will have to solder them first. Check these instructions. In this case, your wires will not have the male pin headers on one end. Do not worry, you can still plug them in the stepper motor hat power terminals.
make instructions for soldering motor wires
Plug in the wheels to the motor as follows (no screws needed):
The Duckiebot is driven by controlling the wheels attached to the DC motors. Still, it requires a βpassiveβ omni-directional wheel (the caster wheel) on the back.
The Magician chassis package contains a steel omni-directional wheel, and the related standoffs and screws to secure it to the chassis-bottom part.
As alternative to omnidirection wheel, caster wheel has less friction. If you have purchased caster wheel, read this section.
To assemble the caster wheel, the following materials are needed:
The lock washers belongs to screw-head side Figure 5.11, i.e. the split lock washer and the flat lock washers stays always near the screw head. The split lock washer stays near the screw head. First split lock washer, then flat lock washer.
Fasten the screws with washers on the caster wheels from the bottom up and screw the metal standoffs from top to bottom. The caster before mounting looks like in Figure 5.13.
Assembly the prepared caster wheels in the front side of duckiebot under the chassis bottom. Fasten the screws with washers from top to bottom.
In order to get all the screws properly into the metal standoffs, let all the screws stay loose within the right positions before all the screws are inserted into the standoffsFigure 5.15.
Put the car upright (omni wheel pointing towards the table) and arrange wires so that they go through the center rectangle. Put 4 spacers with 4 of M3x6 screws on exact position of each corner as below Figure 5.17.
The bottom part of the Duckiebotβs chassis is now ready. The next step is to assemble the Raspberry Pi on chassis-top part.
Before attaching anything to the Raspberry Pi you should add the heat sinks to it. There are 2 small sinks and a big one. The big one best fits onto the processor (the big βBroadcomβ-labeled chip in the center of the top of the Raspberry Pi). One of the small ones can be attached to the small chip that is right next to the Broadcom chip. The third heat sink is optional and can be attached to the chip on the underside of the Raspberry Pi. Note that the chip on the underside is bigger than the heat sink. Just mount the heat sink in the center and make sure all of them are attached tightly.
When this is done fasten the nylon standoffs on the Raspberry Pi, and secure them on the top of the chassis-up part by tightening the nuts on the opposite side of the chassis-up.
Having the Duckiebot image copied in the micro SD card.
Take the micro SD card from the Duckiebox and insert its slot on the Raspberry Pi. The SD card slot is just under the display port, on the short side of the PI, on the flip side of where the header pins are.
If you have camera cables of different lengths available, keep in mind that both are going to work. We suggest to use the longer one, and wrap the extra length under the Raspberry Pi stack.
First, identify the camera cable port on the Pi (between HDMI and power ports) and remove the orange plastic protection (it will be there if the Pi is new) from it. Then, grab the long camera cable (300 mm) and insert in the camera port. To do so, you will need to gently pull up on the black connector (it will slide up) to allow the cable to insert the port. Slide the connector back down to lock the cable in place, making sure it βclicksβ.
insert image with long cable
Make sure the camera cable is inserted in the right direction! The metal pins of the cable should be in contact with the metal terminals in the camera port of the PI.
If you have the long camera cable, the first thing to do is removing the shorter cable that comes with the camera package. Make sure to slide up the black connectors of the camera-camera cable port in order to unblock the cable.
Take the rear part of the camera mount and use it hold the camera in place. Note that the camera is just press-fitted to the camera mount, no screws/nuts are needed.
In case you have not purchased the long camera cable, do not worry! It is still very possible to get a working configuration, but you will have little wiggling space and assembly will be a little harder.
Place the camera on the mount and fasten the camera mount on the chassis-up using M3x10 flathead screws and M3 nuts from the Duckiebox.
Protip: make sure that the camera mount is: (a) geometrically centered on the chassis-up; (b) fastened as forward as it can go; (c) it is tightly fastened. We aim at having a standardized position for the camera and to minimize the wiggling during movement.
If you only have a short camera cable, make sure that the cable is oriented in this direction (text on cable towards the CPU). Otherwise you will have to disassemble the whole thing later. On the long cable the writing is on the other side.
In order to fit the battery, we will need to extend the Magician chassis standoffs with the provided nylon standoff spacers. Grab 4 of them, and secure them to one end of the long metal standoffs provided in the Magician chassis package.
Secure the extended standoff to the 4 corners of the chassis-bottom. The nylon standoffs should smoothly screw in the metal ones. If you feel resistance, donβt force it or the nylon screw might break in the metal standoff. In that case, unscrew the nylon spacer and try again.
Put the battery between the upper and lower decks of the chassis. It is strongly recommended to secure the battery from moving using zip ties.
Figure 5.29 can be taken as an example of how to arrange the long camera cable as well.
Arrange the motor wires through the chassis-up, which will be connected to Stepper Motor HAT later.
Use the provided metal screws from chassis package for fastening the chassis-up above the nylon standoffs instead of the provided M3 nylon screws.
Make sure the GPIO stacking header is carefully aligned with the underlying GPIO pins before applying pressure.
In case with short camera cable, ensure that you doesnβt break the cable while mounting the HAT on the Raspberry Pi. In case with long camera cable,
insert pic with long camera cable
We are using M1 and M2 terminals on the DC motor hat. The left (in robot frame) motor is connected to M1 and the right motor is connected to M2. If you have followed Part A correctly, the wiring order will look like as following pictures:
You are now ready to secure the prepared power wires in Unit C-4 - Preparing the power cable (DB17) to the DC motor HAT power pins.
Connect the battery (not the Raspberry Pi) with the DC motor HAT by making sure you plug the black wire in the pin labeled with a minus: - and the red wire to the plus: + (Figure 4.11).
Fix all the cables on the Duckiebot so that it can run on the way without barrier.
If you have a DB17-Montreal-a or DB17-Chicago-a release, neglect this step and follow the pertinent instructions in Unit E-3 - Assembling the Duckiebot (DB17-lc) regarding the assembly of the PWM hat, its powering through the short angled USB cable, and the power transfer step using a M-M wire.
With each joypad (Figure 5.34) comes a joypad dongle (Figure 5.35). Donβt lose it!
Insert the joypad dongle into one of the USB port of the Raspberry Pi.
Insert 2 AA batteries on the back side of the joypad Figure 5.36.
If we have the bumpers, at what point should we add them?
You shouldnβt have the bumpers at this point. The function of bumpers is to keep the LEDs in place, i.e., they belong to DB17-l configuration. These instructions cover the DB17-jwd configurations. You will find the bumper assembly instructions in Unit E-3 - Assembling the Duckiebot (DB17-lc).
Yeah but I still have the bumpers and am reading this page. So?
The bumpers can be added after the Duckiebot assembly is complete.
I found it hard to mount the camera (the holes werenβt lining up).
Sometimes in life you have to push a little to make things happen. (But donβt push too much or things will break!)
The long camera cable is a bit annoying - I folded it and shoved it in between two hats.
The shorter cable is even more annoying. We suggest wrapping the long camera cable between the chassis and the Raspberry Pi. With some strategic planning, you can use the zipties that keep the battery in place to hold the camera cable in place as well (see figure below-to add)
add pretty cable handling pic
I found that the screwdriver that comes with the chassis kit is too fat to screw in the wires on the hat.
It is possible you got one of the fatter screwdrivers. You will need to figure it out yourself (or ask a TA for help).
I need something to cut the end of the zip tie with.
Scissors typically work out for these kind of jobs (and no, theyβre not provided in a Fall 2017 Duckiebox).
DB17-wjd TTIC)Andrea F. Daniele
Once you have received the parts and soldered the necessary components, it is time to assemble them in a Duckiebot. Here, we provide the assembly instructions for the configuration DB17-wjd (TTIC only).
Duckiebot DB17-wjd parts. The acquisition process is explained in Unit C-2 - Acquiring the parts (DB17-jwd).
Having soldered the DB17-wjd parts. The soldering process is explained in Unit C-3 - Soldering boards (DB17).
Having prepared the power cable. The power cable preparation is explained in Unit C-4 - Preparing the power cable (DB17). Note: Not necessary if you intend to build a DB17-l configuration.
Time: about 30 minutes.
An assembled Duckiebot in configuration DB17-wjd.
The FAQ section at the bottom of this page may already answer some of you comments, questions or doubts.
This section is comprised of 14 parts. Each part builds upon some of the previous parts, so make sure to follow them in the following order.
DB17-wDB17-jDB17-dOpen the Magician Chassis package (Figure 5.1) and take out the following components:
Figure 6.1 shows the components needed to complete this part of the tutorial.
The following video shows how to attach the motors to the bottom plate of the chassis.
Pass the motor holders through the openings in the bottom plate of the chassis as shown in Figure 6.3.
Put the motors between the holders as shown in Figure 6.4.
Orient the motors so that their wires are inwards (i.e., towards the center of the plate).
Use 4 M3x30 screws and 4 M3 nuts to secure the motors to the motor holders. Tighten the screws to secure the holders to the bottom plate of the chassis as shown in Figure 6.5.
Figure 6.6 shows how the motors should be attached to the bottom plate of the chassis.
From the Magician Chassis package take the following components:
Figure 6.7 shows the components needed to complete this part of the tutorial.
The following video shows how to attach the wheels to the motors.
Figure 6.9 shows how the wheels should be attached to the motors.
The Duckiebot is driven by controlling the wheels attached to the DC motors. Still, it requires a passive support on the back. In this configuration an omni-directional wheel is attached to the bottom plate of the chassis to provide such support.
From the Magician Chassis package take the following components:
Figure 6.10 shows the components needed to complete this part of the tutorial.
The following video shows how to attach the omni-directional wheel to the bottom plate of the chassis.
Secure the long spacers to the plate using 2 M3x6 screws and the omni-directional wheel to the spacers using also 2 M3x6 screws as shown in Figure 6.12.
Figure 6.13 shows how the omni-directional wheel should be attached to the plate.
From the Magician Chassis package take the following components:
From the Duckiebot kit take the following components:
Figure 6.14 shows the components needed to complete this part of the tutorial.
The following video shows how to attach the standoffs to the bottom plate of the chassis.
Secure the long metal spacers to the bottom plate using 4 M3x6 screws as shown in Figure 6.16.
Attach the 4 nylon standoffs on top of the metal ones.
Figure 6.17 shows how the standoffs should be attached to the plate.
From the Magician Chassis package take the following components:
From the Duckiebot kit take the following components:
If you have camera cables of different lengths available, keep in mind that both are going to work. We suggest to use the longer one, and wrap the extra length under the Raspberry Pi stack.
Figure 6.18 shows the components needed to complete this part of the tutorial.
The following video shows how to secure the camera to the top plate of the chassis.
If you do not have the 300mm Camera cable you can jump to Step 3.
If you do have the long camera cable, the first thing to do is removing the shorter cable that comes attached to the camera module. Make sure to slide up the black connectors of the camera port on the camera module in order to unblock the cable.
Connect the camera cable to the camera module as shown in Figure 6.20.
Attach the camera module to the camera mount as shown in Figure 6.21.
The camera is just press-fitted to the camera mount, no screws/nuts are needed.
Secure the camera mount to the top plate by using the 2 M3x10 flathead screws and the nuts as shown in Figure 6.22.
Figure 6.23 shows how the camera should be attached to the plate.
From the Duckiebot kit take the following components:
Figure 6.24 shows the components needed to complete this part of the tutorial.
The following video shows how to install the heat sinks on the Raspberry Pi 3.
Remove the protection layer from the heat sinks.
Install the big heat sink on the big βBroadcomβ-labeled integrated circuit (IC).
Install the small heat sink on the small βSMSCβ-labeled integrated circuit (IC).
Figure 6.26 shows how the heat sinks should be installed on the Raspberry Pi 3.
From the Magician Chassis package take the following components:
From the Duckiebot kit take the following components:
Figure 6.27 shows the components needed to complete this part of the tutorial.
The following video shows how to mount the Raspberry Pi 3 on the top plate of the chassis.
Mount 8 M2.5x12 nylon standoffs on the Raspberry Pi 3 as shown in Figure 6.29.
Use the M2.5 nylon hex nuts to secure the Raspberry Pi 3 to the top plate as shown in Figure 6.30.
Figure 6.31 shows how the Raspberry Pi 3 should be mounted on the top plate of the chassis.
From the Magician Chassis package take the following components:
Figure 6.32 shows the components needed to complete this part of the tutorial.
The following video shows how to secure the top plate on top of the bottom plate.
Pass the motor wires through the openings in the top plate.
Use 4 M3x6 screws to secure the top plate to the nylon standoffs (mounted on the bottom plate in Section 6.4 - Chassis standoffs) as shown in Figure 6.34.
Figure 6.35 shows how the top plate should be mounted on the bottom plate.
The power cable preparation is explained in Unit C-4 - Preparing the power cable (DB17).
From the Duckiebot kit take the following components:
DB17)) (1x)Figure 6.36 shows the components needed to complete this part of the tutorial.
The following video shows how to connect the DC Stepper Motor HAT to the Raspberry Pi 3.
Connect the wires of the USB power cable to the terminal block on the DC Stepper Motor HAT labeled as
β5-12V Motor Powerβ as shown in Figure 6.38.
The black wire goes to the negative terminal block (labeled with a minus: -) and the red wire goes to the
positive terminal block (labeled with a plus: +).
Pass the free end of the camera cable through the opening in the DC Stepper Motor HAT as shown in Figure 6.39.
Connect the free end of the camera cable to the CAMERA port on the Raspberry Pi 3 as shown in Figure 6.40.
To do so, you will need to gently pull up on the black connector (it will slide up) to allow the cable to insert the port. Slide the connector back down to lock the cable in place, making sure it βclicksβ.
Make sure the camera cable is inserted in the right direction! The metal pins of the cable must be in contact with the metal terminals in the camera port of the PI. Please be aware that different camera cables have the text on different sides and with different orientation, do not use it as a landmark.
Attach the DC Stepper Motor HAT to the GPIO header on the Raspberry Pi 3. Make sure that the GPIO stacking header of the Motor HAT is carefully aligned with the underlying GPIO pins before applying pressure.
In case you are using a short camera cable, ensure that the camera cable does not stand between the GPIO pins and the the GPIO header socket before applying pressure.
Secure the DC Stepper Motor HAT using 4 M2.5x10 nylon screws.
If you are planning on upgrading your Duckiebot to the configuration DB17-l, you can use 4 M2.5x12
nylon standoffs instead.
Connect the motor wires to the terminal block on the DC Stepper Motor HAT as shown in Figure 6.41.
While looking at the Duckiebot from the back, identify the wires for left and right motor. Connect the left motor wires to the terminals labeled as M1 and the right motor wires to the terminals labeled as M2. This will ensure that the pre-existing software that we will later install on the Duckiebot will send the commands to the correct motors.
Figure 6.42 shows how the DC Stepper Motor HAT should be connected to the Raspberry Pi 3.
From the Duckiebot kit take the following components:
Figure 6.43 shows the components needed to complete this part of the tutorial.
The following video shows how to add the battery to the Duckiebot and turn it on.
Pass the zip tie through the opening in the top plate.
Slide the battery between the two plates. Make sure it is above the zip tie.
Push the free end of the zip tie through the opening in the top plate.
Tighten the zip tie to secure the battery.
Connect the short micro USB cable to the Raspberry Pi 3.
Connect the short micro USB cable to the battery.
Connect the USB power cable to the battery.
Make sure that the LEDs on the Raspberry Pi 3 and the DC Stepper Motor HAT are on.
Figure 6.45 shows how the battery should be installed on the Duckiebot.
DB17-wThis upgrade equips the Duckiebot with a secondary, faster, Wi-Fi connection, ideal for image streaming.
The new configuration is called DB17-w.
Figure 6.46 shows the components needed to complete this upgrade.
DB17-jThis upgrade equips the Duckiebot with manual remote control capabilities. It is particularly
useful for getting the Duckiebot out of tight spots or letting younger ones have a drive, in
addition to providing handy shortcuts to different functions in development phase.
The new configuration is called DB17-j.
Figure 6.48 shows the components needed to complete this upgrade.
The joystick comes with a USB receiver (as shown in Figure 6.48).
HOME button. Make sure that the LED above the SELECT button is steady.
explain how to test the joystick with jstest
DB17-dThis upgrade equips the Duckiebot with an external hard drive that is convenient for storing
videos (logs) as it provides both extra capacity and faster data transfer rates than the microSD
card in the Raspberry Pi 3. Moreover, it is easy to unplug it from the Duckiebot at the end of the
day and bring it over to a computer for downloading and analyzing stored data.
The new configuration is called DB17-d.
Figure 6.50 shows the components needed to complete this upgrade.
If we have the bumpers, at what point should we add them?
You shouldnβt have the bumpers at this point. The function of the bumpers is to keep the LEDs in place, i.e., they belong to DB17-l configuration. These instructions cover the DB17-wjd configurations. You will find the bumper assembly instructions in Unit E-3 - Assembling the Duckiebot (DB17-lc).
Yeah but I still have the bumpers and am reading this page. So?
The bumpers can be added after the Duckiebot assembly is complete.
I found it hard to mount the camera (the holes werenβt lining up).
Sometimes in life you have to push a little to make things happen. (But donβt push too much or things will break!)
The long camera cable is a bit annoying - I folded it and shoved it in between two hats.
The shorter cable is even more annoying. We suggest wrapping the long camera cable between the chassis and the Raspberry Pi. With some strategic planning, you can use the zipties that keep the battery in place to hold the camera cable in place as well (see figure below-to add)
add pretty cable handling pic
I found that the screwdriver that comes with the chassis kit is too fat to screw in the wires on the hat.
It is possible you got one of the fatter screwdrivers. You will need to figure it out yourself (or ask a TA for help).
I need something to cut the end of the zip tie with.
Scissors typically work out for these kind of jobs (and no, theyβre not provided in a Fall 2017 Duckiebox).
These are the instructions to reproduce the Ubuntu image that we use.
Please note that the image is already available, so you donβt need to do this.
However, this documentation is useful if you would like to port the software to a different distribution. Also, we periodically run through these instructions to make sure that they work.
Just in case, letβs re-state this: in Fall 2017, you donβt necessarily need to do the following.
Internet connection to download the packages.
A PC running any Linux with an SD card reader.
Time: about 4 hours (most of it spent waiting for things to download/compile).
A baseline Ubuntu Mate 16.04.2 image with updated software.
Download the image from the page
The file we are looking for is:
filename: ubuntu-mate-16.04.2-desktop-armhf-raspberry-pi.img.xz
size: 1.2 GB
SHA256: dc3afcad68a5de3ba683dc30d2093a3b5b3cd6b2c16c0b5de8d50fede78f75c2
After download, run the command sha256sum to make sure you have the right version:
laptop $ sha256sum ubuntu-mate-16.04.2-desktop-armhf-raspberry-pi.img.xz
dc3afcad68a5de3ba683dc30d2093a3b5b3cd6b2c16c0b5de8d50fede78f75c2
If the string does not correspond exactly, your download was corrupted. Delete the file and try again.
Then decompress using the command xz:
laptop $ xz -d ubuntu-mate-16.04.2-desktop-armhf-raspberry-pi.img.xz
Next, burn the image on to the SD card.
This procedure is explained in Section 17.2 - How to burn an image to an SD card.
Remove the SD card and plug it in again in the laptop.
Ubuntu will mount two partitions, by the name of PI_ROOT and PI_BOOT.
Boot the disk in the Raspberry Pi.
Choose the following options:
language: English
username: ubuntu
password: ubuntu
hostname: duckiebot
Choose the option to log in automatically.
Reboot.
The WiFi was connected to airport network duckietown
with password quackquack.
Afterwards I upgraded all the software preinstalled with these commands:
duckiebot $ sudo apt update
duckiebot $ sudo apt dist-upgrade
Expect dist-upgrade to take quite a long time (up to 2 hours).
The Raspberry Pi is not accessible by SSH by default.
Run raspi-config:
duckiebot $ sudo raspi-config
choose β3. Interfacing Optionsβ, and enable SSH,
We need to enable the camera and the I2C bus.
choose β3. Interfacing Optionsβ, and enable camera, and I2C.
Also disable the graphical boot
choose β2. Boot Optionsβ, configure option for startup. β>B1. Console Text console
Install these packages.
Etckeeper:
duckiebot $ sudo apt install etckeeper
Editors / shells:
duckiebot $ sudo apt install -y vim emacs byobu zsh
Git:
duckiebot $ sudo apt install -y git git-extras
Other:
duckiebot $ sudo apt install htop atop nethogs iftop
duckiebot $ sudo apt install aptitude apt-file
Development:
duckiebot $ sudo apt install -y build-essential libblas-dev liblapack-dev libatlas-base-dev gfortran libyaml-cpp-dev raspberrypi-kernel-headers
Python:
duckiebot $ sudo apt install -y python-dev ipython python-sklearn python-smbus
duckiebot $ sudo apt install -y python-termcolor
duckiebot $ sudo apt install python-frozendict
duckiebot $ sudo apt install python-tables
duckiebot $ pip install comptests
duckiebot $ pip install procgraph
duckiebot $ sudo pip install scipy --upgrade
duckiebot $ sudo pip install ruamel.yaml --upgrade
scipy βupgrade(0.19.1) took about an hour with ethernet connection.
I2C:
duckiebot $ sudo apt install -y i2c-tools
First, mark the kernel packages as not upgradeable:
$ sudo apt-mark hold raspberrypi-kernel raspberrypi-kernel-headers
raspberrypi-kernel set on hold.
raspberrypi-kernel-headers set on hold
Then, download and install the Edimax driver from this repository.
$ git clone git@github.com:duckietown/rtl8822bu.git
$ cd rtl8822bu
$ make
$ sudo make install
Install ROS.
The procedure is given in Section 30.1 - Install ROS.
There are two files that are important to edit.
The file /etc/network/interfaces should look like this:
# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
#source-directory /etc/network/interfaces.d
auto wlan0
# The loopback network interface
auto lo
iface lo inet loopback
# Wireless network interface
allow-hotplug wlan0
iface wlan0 inet dhcp
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
The file /etc/wpa_supplicant/wpa_supplicant.conf should look like this:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
ssid="duckietown"
psk="quackquack"
proto=RSN
key_mgmt=WPA-PSK
pairwise=CCMP
auth_alg=OPEN
}
network={
key_mgmt=NONE
}
The files that describe the network configuration are in the directory
/etc/NetworkManager/system-connections/
This is the contents of the connection file duckietown, which
describes how to connect to the duckietown wireless network:
[connection]
id=duckietown
uuid=e9cef1bd-f6fb-4c5b-93cf-cca837ec35f2
type=wifi
permissions=
secondaries=
timestamp=1502254646
[wifi]
mac-address-blacklist=
mac-address-randomization=0
mode=infrastructure
ssid=duckietown
[wifi-security]
group=
key-mgmt=wpa-psk
pairwise=
proto=
psk=quackquack
[ipv4]
dns-search=
method=auto
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=auto
This is the file
/etc/NetworkManager/system-connections/create-5ghz-network
Contents:
[connection]
id=create-5ghz-network
uuid=7331d1e7-2cdf-4047-b426-c170ecc16f51
type=wifi
# Put the Edimax interface name here:
interface-name=wlx74da38c9caa0 - to change
permissions=
secondaries=
timestamp=1502023843
[wifi]
band=a
# Put the Edimax MAC address here
mac-address=74:DA:38:C9:CA:A0 - to change
mac-address-blacklist=
mac-address-randomization=0
mode=ap
seen-bssids=
ssid=duckiebot-not-configured
[ipv4]
dns-search=
method=shared
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=ignore
Note that there is an interface name and MAC address that need to be changed on each PI.
This enables the SSH server:
$ sudo systemctl enable ssh
Do the following:
Create an empty file using the dd (device-to-device copy) command:
duckiebot $ sudo dd if=/dev/zero of=/swap0 bs=1M count=512
This is for a 512 MB swap space.
Format the file for use as swap:
duckiebot $ sudo mkswap /swap0
Add the swap file to the system configuration:
duckiebot $ sudo vi /etc/fstab
Add this line to the bottom:
/swap0 swap swap
Activate the swap space:
duckiebot $ sudo swapon -a
First, make vi the default editor, using
$ sudo update-alternatives --config editor
and then choose vim.basic.
Then run:
$ sudo visudo
And then change this line:
%sudo ALL=(ALL:ALL) ALL
into this line:
%sudo ALL=(ALL:ALL) NOPASSWD:ALL
You can use the command dpigs to find out which packages take
lots of space.
$ sudo apt install wajig debian-goodies
Either:
$ wajig large
$ dpigs -H -n 20
Stuff to remove:
$ sudo apt remove thunderbird
$ sudo apt remove libreoffice-\*
$ sudo apt remove openjdk-8-jre-headless
$ sudo apt remove fonts-noto-cjk
$ sudo apt remove brasero
At the end, remove extra dependencies:
$ sudo apt autoremove
And remove the apt cache using:
$ sudo apt clean
The total size should be around 6.6GB.
You should make the ubuntu user belong to the i2c and input groups:
duckiebot $ sudo adduser ubuntu i2c
duckiebot $ sudo adduser ubuntu input
duckiebot $ sudo adduser ubuntu video
You may need to do the following (but might be done already through raspi-config): XXX
duckiebot $ sudo udevadm trigger
Do the basic SSH config.
The procedure is documented in Section 19.3 - Local configuration.
this is not in the aug10 image.
Add .authorized_keys so that we can all do passwordless SSH.
The key is at the URL
https://www.dropbox.com/s/pxyou3qy1p8m4d0/duckietown_key1.pub?dl=1
Download to .ssh/authorized_keys:
duckiebot $ curl -o .ssh/authorized_keys URL above
Add the following lines to ~ubuntu/.bashrc:
echo ""
echo "Welcome to a duckiebot!"
echo ""
echo "Reminders:"
echo ""
echo "1) Do not use the user 'ubuntu' for development - create your own user."
echo "2) Change the name of the robot from 'duckiebot' to something else."
echo ""
export EDITOR=vim
At this point, before you copy/distribute the image, create a user,
install the software, and make sure that what-the-duck does not complain
about any missing package.
(Ignore what-the-duckβs errors about things that are not set up yet, like
users.)
You may now want to create an image that you can share with your friends. They will think you are cool because they wonβt have to duplicate all of the work that you just did. Luckily this is easy. Just power down the duckiebot with:
duckiebot $ sudo shutdown -h now
and put the SD card back in your laptop.
The procedure of how to burn an image is explained in
Section 17.2 - How to burn an image to an SD card; except you will invert the if and of destinations.
You may want to subsequently shrink the image, for example if your friends have smaller SD cards than you.
The procedure of how to shrink an image is explained in Section 17.3 - How to shrink an image.
We should install Git LFS on the Raspberry Pi, but so far AC did not have any luck. See Section 28.1 - Generic installation instructions.
Andrea
Before you prepare the Duckiebot, you need to have a laptop with Ubuntu installed.
A laptop with free disk space.
Internet connection to download the Ubuntu image.
About 30 minutes.
A laptop ready to be used for Duckietown.
Install Ubuntu 16.04.3.
For instructions, see for example this online tutorial.
On the choice of username: During the installation, create a user for yourself with a username different from ubuntu, which is the default. Otherwise, you may get confused later.
Use etckeeper to keep track of the configuration in /etc:
laptop $ sudo apt install etckeeper
Install ssh to login remotely and the server:
laptop $ sudo apt install ssh
Use byobu:
laptop $ sudo apt install byobu
Use vim:
laptop $ sudo apt install vim
Use htop to monitor CPU usage:
laptop $ sudo apt install htop
Additional utilities for git:
laptop $ sudo apt install git git-extras
Other utilities:
laptop $ sudo apt install avahi-utils ecryptfs-utils
Install ROS on your laptop.
The procedure is given in Section 30.1 - Install ROS.
This is Flux for Linux. It is an accessibility/lab safety issue: bright screens damage eyes and perturb sleep [6].
Install redshift and run it.
laptop $ sudo apt install redshift-gtk
Set to βautostartβ from the icon (on the panel - near wifi/lan).
sudoSet up passwordless sudo.
This procedure is described in Section 7.11 - Passwordless sudo.
Huh I donβt know - this is great for usability, but horrible for security. If you step away from your laptop for a second and donβt lock the screen, a nasty person could sudo rm -rf /. -FG
Do the basic SSH config.
The procedure is documented in Section 19.3 - Local configuration.
usernameNext, create a private/public key pair for the user; call it username@robot name.
The procedure is documented in Section 19.5 - Creating an SSH keypair.
usernameβs public key to GithubAdd the public key to your Github account.
The procedure is documented in Section 29.3 - Add a public key to Github.
If the step is done correctly, this command should succeed:
duckiebot $ ssh -T git@github.com
Set up Git locally.
The procedure is described in Section 27.3 - Setting up global configurations for Git.
Optional but very encouraged: install the duckuments system. This will allow you to have a local copy of the documentation and easily submit questions and changes.
The procedure is documented in Section 1.3 - Installing the documentation system.
Andrea
An SD card of dimensions at least 16 GB.
A computer with an internet connection, an SD card reader, and 16 GB of free space.
An assembled Duckiebot in configuration DB17. This is the result of Unit C-5 - Assembling the Duckiebot (DB17-jwd).
A Duckiebot that is configured correctly, that you can connect to with your laptop and hopefully also has internet access
On the laptop, download the compressed image at this URL:
https://www.dropbox.com/s/ckpqpp0cav3aucb/duckiebot-RPI3-AD-2017-09-12.img.xz?dl=1
The size is 1.7 GB.
You can use:
$ wget -O duckiebot-RPI3-AD-2017-09-12.img.xz URL above
The original was:
$ curl -o duckiebot-RPI3-AD-2017-09-12.img.xz URL above
It looks like that curl cannot be used with Drobpox links because it does not follow redirects.
To make sure that the image is downloaded correctly, compute its hash
using the program sha256sum:
$ sha256sum duckiebot-RPI3-AD-2017-09-12.img.xz
7136f9049b230de68e8b2d6df29ece844a3f830cc96014aaa92c6d3f247b6130 duckiebot-RPI3-AD-2017-09-12.img.xz
Compare the hash that you obtain with the hash above. If they are different, there was some problem in downloading the image.
Uncompress the file:
$ xz -d -k --verbose duckiebot-RPI3-AD-2017-09-12.img.xz
This will create a file of 11 GB in size.
Next, burn the image on disk.
The procedure of how to burn an image is explained in Section 17.2 - How to burn an image to an SD card.
Put the SD Card in the Duckiebot.
Turn on the Duckiebot by connecting the power cable to the battery.
Add figure
In general, for the battery: if itβs off, a single click on the power button will turn the battery on. When itβs on, a single click will show you the charge indicator (4 white lights = full), and holding the button for 3s will turn off the battery. Shutting down the Duckiebot is not recommended because it may cause corruption of the SD card.
You can login to the Duckiebot in two ways:
duckietown WiFi network.In the worst case, you can use an HDMI monitor and a USB keyboard.
Connect the Duckiebot and your laptop to the same network switch.
Allow 30 s - 1 minute for the DHCP to work.
The Duckiebot connects automatically to a 2.4 GHz network
called βduckietownβ and password βquackquackβ.
Connect your laptop to the same wireless network.
To test that the Duckiebot is connected, try to ping it.
The hostname of a freshly-installed duckiebot is duckiebot-not-configured:
laptop $ ping duckiebot-not-configured.local
You should see output similar to the following:
PING duckiebot-not-configured.local (X.X.X.X): 56 data bytes
64 bytes from X.X.X.X: icmp_seq=0 ttl=64 time=2.164 ms
64 bytes from X.X.X.X: icmp_seq=1 ttl=64 time=2.303 ms
...
Next, try to log in using SSH, with account ubuntu:
laptop $ ssh ubuntu@duckiebot-not-configured.local
The password is ubuntu.
By default, the robot boots into Byobu.
Please see Unit J-26 - Byobu for an introduction to Byobu.
Not sure itβs a good idea to boot into Byobu. -??
Next, we need to update to bring the system up to date.
Use these commands
duckiebot $ sudo apt update
duckiebot $ sudo apt dist-upgrade
It is now time to give a name to the Duckiebot.
These are the criteria:
-β, β_β, etc.) .duckiebotβ, βrobotβ or similar.From here on, we will refer to this string as βrobot nameβ.
Every time you see robot name, you should substitute the name that you chose.
We will put the robot name in configuration files.
Files in /etc are only writable by root,
so you need to use sudo to edit them. For example:
duckiebot $ sudo vi filename
Edit the file
/etc/hostname
and put βrobot nameβ instead of duckiebot-not-configured.
Also edit the file
/etc/hosts
and put βrobot nameβ where duckiebot-not-configured appears.
The first two lines of /etc/hosts should be:
127.0.0.1 localhost
127.0.1.1 robot name
there is a command hostname that promises to change the hostname.
However, the change given by that command does not persist across reboots. You
need to edit the files above for the changes to persist.
Never add other hostnames in /etc/hosts. It is a tempting
fix when DNS does not work, but it will cause other problems
subsequently.
Then reboot the Raspberry Pi using the command
$ sudo reboot
After reboot, log in again, and run the command hostname to check that the
change has persisted:
$ hostname
robot name
If your SD card is larger than the image, youβll want to expand the filesystem on your robot so that you can use all of the space available. Achieve this with:
duckiebot $ sudo raspi-config --expand-rootfs
and then reboot
duckiebot $ sudo shutdown -r now
once rebooted you can test whether this was successful by doing
duckiebot $ df -lh
the output should give you something like:
Filesystem Size Used Avail Use% Mounted on
/dev/root 15G 6.3G 8.2G 44% /
devtmpfs 303M 0 303M 0% /dev
tmpfs 431M 0 431M 0% /dev/shm
tmpfs 431M 12M 420M 3% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 431M 0 431M 0% /sys/fs/cgroup
/dev/mmcblk0p1 63M 21M 43M 34% /boot
tmpfs 87M 0 87M 0% /run/user/1000
You should see that the Size of your /dev/root Filesystem is βcloseβ to the size of your SD card.
You must not use the ubuntu user for development.
Instead, you need to create a new user.
Choose a user name, which we will refer to as username.
To create a new user:
duckiebot $ sudo useradd -m username
Make the user an administrator by adding it to the group sudo:
duckiebot $ sudo adduser username sudo
Make the user a member of the groups input, video, and i2c
duckiebot $ sudo adduser username input
duckiebot $ sudo adduser username video
duckiebot $ sudo adduser username i2c
Set the shell bash:
duckiebot $ sudo chsh -s /bin/bash username
To set a password, use:
duckiebot $ sudo passwd username
At this point, you should be able to login to the new user from the laptop using the password:
laptop $ ssh username@robot name
Next, you should repeat some steps that we already described.
What steps?? -LP
Do the basic SSH config.
The procedure is documented in Section 19.3 - Local configuration.
usernameNext, create a private/public key pair for the user; call it username@robot name.
The procedure is documented in Section 19.5 - Creating an SSH keypair.
Once you have your SSH key pair on both your laptop and your Duckiebot, as well as your new user- and hostname set up on your Duckiebot, then you should set up an SSH alias as described in Section 15.1 - SSH aliases. This allows your to log in for example with
laptop $ ssh abc
instead of
laptop $ ssh username@robot name
where you can chose abc to be any alias / shortcut.
usernameβs public key to GithubAdd the public key to your Github account.
The procedure is documented in Section 29.3 - Add a public key to Github.
If the step is done correctly, the following command should succeed and give you a welcome message:
duckiebot $ ssh -T git@github.com
Hi username! You've successfully authenticated, but GitHub does not provide shell access.
This procedure is in Section 27.3 - Setting up global configurations for Git.
Make sure that you can login passwordlessly to your user from the laptop.
The procedure is explained in Section 19.6 - How to login without a password.
In this case, we have:
local = laptop, local-user = your local user on the laptop,
remote = robot name, remote-user = username.
If the step is done correctly, you should be able to login from the laptop to the robot, without typing a password:
laptop $ ssh username@robot name
In general, if you find yourself:
ssh more than onceit means you should learn more about Linux and networks, and you are setting yourself up for failure.
Yes, you βcan do withoutβ, but with an additional 30 seconds of your time. The 30 seconds you are not saving every time are the difference between being productive roboticists and going crazy.
Really, it is impossible to do robotics when you have to think about IPs and passwordsβ¦
If you know what you are doing, you are welcome to install and use additional shells, but please keep Bash as be the default shell. This is important for ROS installation.
For the record, our favorite shell is ZSH with oh-my-zsh.
Check that the camera is connected using this command:
duckiebot $ vcgencmd get_camera
supported=1 detected=1
If you see detected=0, it means that the hardware connection is not working.
You can test the camera right away using a command-line utility
called raspistill.
Use the raspistill command to capture the file out.jpg:
duckiebot $ raspistill -t 1 -o out.jpg
Then download out.jpg to your computer using scp for inspection.
For instructions on how to use scp, see Subsection 21.1.1 - Download a file with SCP.
In order to show that your Duckiebot is ready for the task of driving around happy little duckies, the robot has to fly the Duckietown flag. When you are still logged in to the Duckiebot you can download and install the banner like this:
Download the ANSI art file from Github:
duckiebot $ wget --no-check-certificate -O duckie.art "https://raw.githubusercontent.com/duckietown/Software/master/misc/duckie.art"
(optional) If you want, you can preview the logo by just outputting it onto the command line:
duckiebot $ cat duckie.art
Next up create a new empty text file in your favorite editor and add the code for showing your duckie pride:
Letβs say I use nano, I open a new file:
duckiebot $ nano 20-duckie
And in there I add the following code (which by itself just prints the duckie logo):
#!/bin/sh
printf "\n$(cat /etc/update-motd.d/duckie.art)\n"
Then save and close the file. Finally you have to make this file executableβ¦
duckiebot $ chmod +x 20-duckie
β¦and copy both the duckie logo and the script into a specific directory /etc/update-motd.d to make it appear when you login via SSH. motd stands for βmessage of the dayβ. This is a mechanism for system administrators to show users news and messages when they login. Every executable script in this directory which has a filename a la NN-some name will get exected when a user logs in, where NN is a two digit number that indicates the order.
sudo cp duckie.art /etc/update-motd.d
sudo cp 20-duckie /etc/update-motd.d
Finally log out of SSH via exit and log back in to see duckie goodness.
detected=0
If you see detected=0, it is likely that the camera is not connected correctly.
If you see an error that starts like this:
mmal: Cannot read camera info, keeping the defaults for OV5647
...
mmal: Camera is not detected. Please check carefully the camera module is installed correctly.
then, just like it says: βPlease check carefully the camera module is installed correctly.β.
random wget, curl, git, and apt calls fail with SSL errors.
Thatβs probably actually an issue with your system time. Type the command timedatectl into a terminal, hit enter and see if the time is off. If it is, you might want to follow the intructions from this article,
or entirely uninstall your NTP service and manually grab the time on reboot. Itβs a bit dirty, but works surprisingly well.
Cannot find /etc folder for configuring the Wi-Fi. I only see Desktop, Downloads when starting up the Duckiebot.
If a directory name starts with /, itβs not supposed to be in the home directory, but rather at the root of the filesystem. You are currently in /home/ubuntu. Type ls / to see the folders at the root, including `/etc.
A Duckiebot in configuration DB17-CO+w
Either a router that you have control over that has internet access, or your credentials for connecting to an existing wireless network
Patience (channel your inner Yoda)
A Duckiebot that you can connect to and that is connected to the internet
this page is primarily for folks operating with the βtwo-networkβ configuration, C0+w. For a one adapter setup you will can skip directly to Section 10.2 - Setting up wireless network configuration, but you will have to connect to a network that you can ssh through.
The basic idea is that we are going to use the βEdimaxβ thumbdrive adapter to create a dedicated wireless network that you can always connect to with your laptop. Then we are going to use the built-in Broadcom chip on the Pi to connect to the internet, and then the network will be bridged.
DB17-w) Configure the robot-generated networkThis part should work every time with very low uncertainty.
The Duckiebot in configuration C0+w can create a WiFi network.
It is a 5 GHz network; this means that you need to have a 5 GHz WiFi adapter in your laptop.
First, make sure that the Edimax is correctly installed.
Using iwconfig, you should see four interfaces:
duckiebot $ iwconfig
wlxAABBCCDDEEFFGG unassociated Nickname:"rtl8822bu"
...
lo no wireless extensions.
enxb827eb1f81a4 no wireless extensions.
wlan1 IEEE 802.11bgn ESSID:"duckietown"
...
Make note of the name wlxAABBCCDDEEFFGG.
Look up the MAC address using the command:
duckiebot $ ifconfig wlxAABBCCDDEEFFGG
wlxAABBCCDDEEFFGG Link encap:Ethernet HWaddr AA:BB:CC:DD:EE:FF:GG
Then, edit the connection file
/etc/NetworkManager/system-connections/create-5ghz-network
Make the following changes:
interface-name=..., put βwlxAABBCCDDEEFFGGβ.mac-address=..., put βAA:BB:CC:DD:EE:FF:GGβ.ssid=duckiebot-not-configured, put βssid=robot nameβ.![Newly upgraded]
To ensure nobody piggybacks on our connection, which poses a security risk especially in a public environment, we will protect access to the 5 GHz WiFi through a password. To set a password you will need to log in the Duckiebot with the default βubuntuβ username and password and change your system files. In the /etc/NetworkManager/system-connections/create-5ghz-network, add:
[wifi-security]
key-mgmt=wpa-psk
psk=YOUR_OWN_WIFI_PASSWORD_NO_QUOTATION_MAKRS_NEEDED
auth-alg=open
and then reboot.
At this point you should see a new network being created named βrobot nameβ, protected by the password you just set.
Make sure the password contains min. 8 character or combined with numbers. If no networks shows up after the configuration and no feedback from system, please check the content of the file again. The program, which activates the Edimax wifi adapter is very sensitive to its content.
Adding a password to your 5GHz connection is a mandatory policy in the Zurich branch.
You are connected to the Duckiebot via WiFi, but the Duckiebot also needs to connect to the internet in order to get updates and install some software. This part is a little bit more of a βblack artβ since we cannot predict every possible network configurations. Below are some settings that have been verified to work in different situations:
duckietown WiFiCheck with your phone or laptop if there is a WiFi in reach with the name of duckietown. If there is, you are all set. The defaut configuration for the Duckiebot is to have one WiFi adapter connect to this network and the other broadcast the access point which you are currently connected to.
eduroam WiFi (Non-UdeM/McGill instructions)If there should be no duckietown network in reach then you have to manually add a network configuration file for the network that youβd like to connect to. Most universities around the world should have to eduroam network available. You can use it for connecting your Duckiebot.
Save the following block as new file in /etc/NetworkManager/system-connections/eduroam:
[connection]
id=eduroam
uuid=38ea363b-2db3-4849-a9a4-c2aa3236ae29
type=wifi
permissions=user:oem:;
secondaries=
[wifi]
mac-address=the MAC address of your internal wifi adapter, wlan0
mac-address-blacklist=
mac-address-randomization=0
mode=infrastructure
seen-bssids=
ssid=eduroam
[wifi-security]
auth-alg=open
group=
key-mgmt=wpa-eap
pairwise=
proto=
[802-1x]
altsubject-matches=
eap=ttls;
identity=your eduroam username@your eduroam domain
password=your eduroam password
phase2-altsubject-matches=
phase2-auth=pap
[ipv4]
dns-search=
method=auto
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto
Set the permissions on the new file to 0600.
sudo chmod 0600 /etc/NetworkManager/system-connections/eduroam
eduroam WiFi (UdeM/McGill instructions)Save the following block as new file in /etc/NetworkManager/system-connections/eduroam-USERNAME:
where USERNAME is the your logged-in username in the duckiebot.
[connection]
id=eduroam
uuid=38ea363b-2db3-4849-a9a4-c2aa3236ae29
type=wifi
permissions=user:USERNAME:;
secondaries=
[wifi]
mac-address=the MAC address of your internal wifi adapter, wlan0
mac-address-blacklist=
mac-address-randomization=0
mode=infrastructure
seen-bssids=
ssid=eduroam
[wifi-security]
auth-alg=open
group=
key-mgmt=wpa-eap
pairwise=
proto=
[802-1x]
altsubject-matches=
eap=peap;
identity=DGTIC UNIP
password=DGTIC PWD
phase2-altsubject-matches=
phase2-auth=mschapv2
[ipv4]
dns-search=
method=auto
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto
Set the permissions on the new file to 0600.
sudo chmod 0600 /etc/NetworkManager/system-connections/eduroam-USERNAME
UdeM avec cryptagesomeone replicate please - LP
your can use the autoconnect-priority=XX inside the [connection] block to establish a priority. If you want to connect to one network preferentially if two are available then give it a higher priority.
Save the following block as new file in /etc/NetworkManager/system-connections/secure:
[connection]
id=secure
uuid=e9cef1bd-f6fb-4c5b-93cf-cca837ec35f2
type=wifi
permissions=
secondaries=
timestamp=1502254646
autoconnect-priority=100
[wifi]
mac-address-blacklist=
mac-address-randomization=0
mode=infrastructure
ssid=UdeM avec cryptage
security=wifi-security
[wifi-security]
key-mgmt=wpa-eap
[802-1x]
eap=peap;
identity=DGTIC UNIP
phase2-auth=mschapv2
password=DGTIC PWD
[ipv4]
dns-search=
method=auto
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=auto
Set the permissions on the new file to 0600.
sudo chmod 600 /etc/NetworkManager/system-connections/secure
First run the following to see what networks are available:
duckiebot $ nmcli dev wifi list
You should see the network that you are trying to connect (SSID)) to and you should know the password. To connect to it run:
duckiebot $ sudo nmcli dev wifi con SSID password PASSWORD
The following instructions will lead you to connect your PI to the βethβ wifi network.
First, run the following on duckiebot
duckiebot $ iwconfig
...
lo no wireless extensions.
enxbxxxxxxxxxxx no wireless extensions.
...
Make note of the name enxbxxxxxxxxxxx. xxxxxxxxxxx should be a string that has 11 characters that is formed by numbers and lower case letters.
Second, edit the file /etc/network/interfaces which requires sudo so that it looks like the following, and make sure the enxbxxxxxxxxxxx matches.
Pay special attention on the line βpre-up wpa_supplicant -B -D wext -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.confβ.This is expected to be exactly one line instead of two but due to formatting issue it is shown as two lines.
Also, make sure every characters match exactly with the provided ones. TAs will not help you to do spelling error check.
# interfaces(5) file used by ifup(8) and ifdown(8) Include files from /etc/network/ interfaces.d:
source-directory /etc/network/interfaces.d
# The loopback network interface
auto lo
auto enxbxxxxxxxxxxx
# the wired network setting
iface enxbxxxxxxxxxxx inet dhcp
# the wireless network setting
auto wlan0
allow-hotplug wlan0
iface wlan0 inet dhcp
pre-up wpa_supplicant -B -D wext -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf
post-down killall -q wpa_supplicant
Third, edit the file /etc/wpa_supplicant/wpa_supplicant.conf which requires sudo so that it looks like the following, and make sure you substitute [identity] and [password] content with your eth account information:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
ssid="eth"
key_mgmt=WPA-EAP
group=CCMP TKIP
pairwise=CCMP TKIP
eap=PEAP
proto=RSN
identity="your user name goes here"
password="your password goes here"
phase1="peaplabel=0"
phase2="auth=MSCHAPV2"
priority=1
}
Fourth, reboot your PI.
duckiebot $ sudo reboot
Then everything shall be fine. The PI will connect to βethβ automatically everytime it starts.
Laptop configured, according to Unit C-8 - Installing Ubuntu on laptops.
You have configured the Duckiebot. The procedure is documented in Unit C-9 - Duckiebot Initialization.
You have created a Github account and configured public keys, both for the laptop and for the Duckiebot. The procedure is documented in Unit J-29 - Setup Github access.
You can run the joystick demo.
Clone the repository in the directory ~/duckietown:
duckiebot $ git clone git@github.com:duckietown/Software.git ~/duckietown
For the above to succeed you should have a Github account already set up.
It should not ask for a password.
you must not clone the repository using the URL starting with https. Later steps will fail.
It asks for a password.
You missed some of the steps described in Unit J-29 - Setup Github access.
Other weird errors.
Probably the time is not set up correctly. Use ntpdate as above:
$ sudo ntpdate -u us.pool.ntp.org
Or see the hints in the troubleshooting section on the previous page.
The software used for the Duckiebots changes every day, this means that also the dependencies change. In order to check whether your system meets all the requirements for running the software and install all the missing packages (if any), we can run the following script:
duckiebot $ cd ~/duckietown
duckiebot $ /bin/bash ./dependencies_for_duckiebot.sh
This command will install only the packages that are not already installed in your system.
All the following commands should be run in the ~/duckietown directory:
duckiebot $ cd ~/duckietown
Now we are ready to make the workspace. First you need to source the baseline ROS environment:
duckiebot $ source /opt/ros/kinetic/setup.bash
Then, build the workspace using:
duckiebot $ catkin_make -C catkin_ws/
For more information about catkin_make, see Section 30.6 - catkin_make.
there is a known bug, for which it fails the first time on the Raspberry Pi. Try again; it will work.
I got no error on first execution on the Raspberry Pi
Clone the relevant duckiefleet repository into ~/duckiefleet.
See see Subsection 4.1.2 - Duckiefleet directory DUCKIEFLEET_ROOT to find the right duckiefleet repository.
In ~/.bashrc set DUCKIEFLEET_ROOT to point to the directory:
export DUCKIEFLEET_ROOT=~/duckiefleet
Also, make sure that you execute ~/.bashrc in the current shell by running the command:
source ~/.bashrc
Next, you need to add your robot to the vehicles database. This is not optional and required in order to launch any ROS scripts.
You have already a copy of the vehicles database in the folder robots of DUCKIEFLEET_ROOT.
Copy the file emma.robot.yaml to robotname.robot.yaml, where robotname
is your robotβs hostname. Then edit the copied file to represent your Duckiebot.
For information about the format, see Section 4.2 - The βscuderiaβ (vehicle database).
Generate the machines file.
The procedure is listed here: Section 4.3 - The machines file.
Finally, push your robot configuration to the duckiefleet repo.
Plug the joystick receiver in one of the USB port on the Raspberry Pi.
To make sure that the joystick is detected, run:
duckiebot $ ls /dev/input/
and check if there is a device called js0 on the list.
Make sure that your user is in the group input and i2c:
duckiebot $ groups
username sudo input i2c
If input and i2c are not in the list, you missed a step. Ohi ohi!
You are not following the instructions carefully!
Consult again Section 9.11 - Create your user.
To test whether or not the joystick itself is working properly, run:
duckiebot $ jstest /dev/input/js0
Move the joysticks and push the buttons. You should see the data displayed change according to your actions.
SSH into the Raspberry Pi and run the following from the duckietown directory:
duckiebot $ cd ~/duckietown
duckiebot $ source environment.sh
The environment.sh setups the ROS environment at the terminal (so you can use
commands like rosrun and roslaunch).
Now make sure the motor shield is connected.
Run the command:
duckiebot $ roslaunch duckietown joystick.launch veh:=robot name
If there is no βredβ output in the command line then pushing the left joystick knob controls throttle - right controls steering.
This is the expected result of the commands:
| left joystick up | forward |
| left joystick down | backward |
| right joystick left | turn left (positive yaw) |
| right joystick right | turn right (negative yaw) |
It is possible you will have to unplug and replug the joystick or just push lots of buttons on your joystick until it wakes up. Also make sure that the mode switch on the top of your joystick is set to βXβ, not βDβ.
XXX Is all of the above valid with the new joystick?
Close the program using Ctrl-C.
The robot moves weirdly (e.g. forward instead of backward).
The cables are not correctly inserted. Please refer to the assembly guide for pictures of the correct connections. Try swapping cables until you obtain the expected behavior.
Check that the joystick has the switch set to the position βxβ.And the mode light should be off.
The left joystick does not work.
If the green light on the right to the βmodeβ button is on, click the βmodeβ button to turn the light off. The βmodeβ button toggles between left joystick or the cross on the left.
The robot does not move at all.
The cables are disconnected.
The program assumes that the joystick is at /dev/input/js0.
In doubt, see Section 11.6 - Test that the joystick is detected.
Generally speaking, you can terminate any roslaunch command with Ctrl-C.
To completely shutdown the robot, issue the following command:
duckiebot $ sudo shutdown -h now
Then wait 30 seconds.
If you disconnect the power before shutting down properly using shutdown,
the system might get corrupted.
Then, disconnect the power cable, at the battery end.
As an alternative you can use the poweroff command:
duckiebot $ sudo poweroff
If you disconnect frequently the cable at the Raspberry Piβs end, you might damage the port.
You have configured the Duckiebot. The procedure is documented in Unit C-9 - Duckiebot Initialization.
You know the basics of ROS (launch files, roslaunch, topics, rostopic).
put reference
You know that the camera works under ROS.
It might be useful to do a quick camera hardware check.
The procedure is documented in Section 9.13 - Hardware check: camera.
On the laptop, create two Byobu windows.
A quick reference about Byobu commands is in Unit J-26 - Byobu.
You will use the two windows as follows:
You could also use multiple terminals instead of one terminal with multiple Byobu windows. However, using Byobu is the best practice to learn.
In the first window, we will launch the nodes that control the camera.
All the following commands should be run in the ~/duckietown directory:
duckiebot $ cd ~/duckietown
Activate ROS:
duckiebot $ source environment.sh
Run the launch file called camera.launch:
duckiebot $ roslaunch duckietown camera.launch veh:=robot name
At this point, you should see the red LED on the camera light up continuously.
In the terminal you should not see any red message, but only happy messages like the following:
...
[INFO] [1502539383.948237]: [/robot name/camera_node] Initialized.
[INFO] [1502539383.951123]: [/robot name/camera_node] Start capturing.
[INFO] [1502539384.040615]: [/robot name/camera_node] Published the first image.
For more information about roslaunch and βlaunch filesβ, see Section 30.3 - roslaunch.
Switch to the second window.
All the following commands should be run in the ~/duckietown directory:
duckiebot $ cd ~/duckietown
Activate the ROS environment:
duckiebot $ source environment.sh
You can see a list of published topics with the command:
duckiebot $ rostopic list
For more information about rostopic, see Section 30.5 - rostopic.
You should see the following topics:
/robot name/camera_node/camera_info
/robot name/camera_node/image/compressed
/robot name/camera_node/image/raw
/rosout
/rosout_agg
You can use rostopic hz to see the statistics about the publishing frequency:
duckiebot $ rostopic hz /robot name/camera_node/image/compressed
On a Raspberry Pi 3, you should see a number close to 30 Hz:
average rate: 30.016
min: 0.026s max: 0.045s std dev: 0.00190s window: 841
You can view the messages in real time with the command rostopic echo:
duckiebot $ rostopic echo /robot name/camera_node/image/compressed
You should see a large sequence of numbers being printed to your terminal.
Thatβs the βimageβ β as seen by a machine.
If you are Neo, then this already makes sense. If you are not Neo, in
Unit C-14 - RC+camera remotely, you will learn how to visualize the image stream
on the laptop using rviz.
use Ctrl-C to stop rostopic.
Physically focus the camera.
Andrea
You can run the joystick demo from the Raspberry Pi. The procedure is documented in Unit C-11 - Software setup and RC remote control.
You can run the joystick demo from your laptop.
ROS nodes can be launched in two ways:
Which is better when is a long discussion that will be done later. Here we set up the βremote launchβ.
draw diagrams
Software repository on the laptopAs you did on the Duckiebot, you should clone the Software
repository in the ~/duckietown directory.
The procedure is documented in Section 11.1 - Clone the Duckietown repository.
Then, you should build the repository.
This procedure is documented in Section 11.3 - Set up the ROS environment on the Duckiebot.
In a previous step you have created a robot configuration file and pushed it to the duckiefleet repo. Now you have to pull duckiefleet on the laptop and rebuild the machines configuration file there.
The procedure is documented in Section 11.5 - Add your vehicle data to the robot database.
Now you are ready to launch the joystick demo remotely.
Make sure that you can login with SSH without a password. From the laptop, run:
laptop $ ssh username@robot name.local
If this doesnβt work, you missed some previous steps.
Run this on the laptop:
laptop $ source environment.sh
laptop $ roslaunch duckietown joystick.launch veh:=robot name
You should be able to drive the vehicle with joystick just like the last example. Note that remotely launching nodes from your laptop doesnβt mean that the nodes are running on your laptop. They are still running on the Raspberry Pi in this case.
For more information about roslaunch, see Section 30.3 - roslaunch.
rqt_consoleAlso, you might have notice that the terminal where you launch the launch file is not printing all the printouts like the previous example. This is one of the limitation of remote launch.
Donβt worry though, we can still see the printouts using rqt_console.
On the laptop, open a new terminal window, and run:
laptop $ export ROS_MASTER_URI=http://robot name.local:11311/
laptop $ rqt_console
You should see a nice interface listing all the printouts in real time,
completed with filters that can help you find that message you are looking for
in a sea of messages.
If rqt_console does not show any message, check out the Troubleshooting section below.
You can use Ctrl-C at the terminal where roslaunch was executed to stop all the
nodes launched by the launch file.
For more information about rqt_console, see Section 30.2 - rqt_console.
rqt_console does not show any message.
Open rqt_console. Go to the Setup window (top-right corner).
Change the βRosout Topicβ field from /rosout_agg to /rosout. Confirm.
roslaunch fails with an error similar to the following:
remote[robot name.local-0]: failed to launch on robot name:
Unable to establish ssh connection to [username@robot name.local:22]:
Server u'robot name.local' not found in known_hosts.
You have not followed the instructions that told you to add the HostKeyAlgorithms
option. Delete ~/.ssh/known_hosts and fix your configuration.
The procedure is documented in Section 19.3 - Local configuration.
Andrea
You can run the joystick demo remotely. The procedure is documented in Unit C-13 - RC control launched remotely.
You can read the camera data from ROS. The procedure is documented in Unit C-12 - Reading from the camera.
You know how to get around in Byobu. You can find the Byobu tutorial in Unit J-26 - Byobu.
You can run the joystick demo from your laptop and see the camera image on the laptop.
We are assuming that the joystick demo in Unit C-13 - RC control launched remotely worked.
We are assuming that the procedure in Unit C-12 - Reading from the camera succeeded.
We also assume that you terminated all instances of roslaunch with
Ctrl-C, so that currently there is nothing running in any
window.
On the laptop, this time create four Byobu windows.
A quick reference about Byobu commands is in Unit J-26 - Byobu.
You will use the four windows as follows:
rviz to see the camera image.Add figures
In the first window, launch the joystick remotely using the same procedure in Section 13.4 - Start the demo.
laptop $ source environment.sh
laptop $ roslaunch duckietown joystick.launch veh:=robot name
You should be able to drive the robot with the joystick at this point.
In the second window, we will launch the nodes that control the camera.
The launch file is called camera.launch:
laptop $ source environment.sh
laptop $ roslaunch duckietown camera.launch veh:=robot name
You should see the red led on the camera light up.
It is recommended to launch the joystick and the camera from onboard the robot after sshing in - LP
Open a third terminal on the laptop.
You can see a list of topics currently on the ROS_MASTER with the commands:
laptop $ source environment.sh
laptop $ export ROS_MASTER_URI=http://robot name.local:11311/
laptop $ rostopic list
You should see the following:
/diagnostics
/robot name/camera_node/camera_info
/robot name/camera_node/image/compressed
/robot name/camera_node/image/raw
/robot name/joy
/robot name/wheels_driver_node/wheels_cmd
/rosout
/rosout_agg
rvizLaunch rviz by using these commands:
laptop $ source environment.sh
laptop $ source set_ros_master.sh robot name
laptop $ rviz
For more information about rviz, see Section 30.4 - rviz.
In the rviz interface, click βAddβ on the lower left, then the βBy topicβ
tag, then select the βImageβ topic by the name
/robot name/camera_node/image/compressed
Then click βokβ. You should be able to see a live stream of the image from the camera.
To stop the nodes: You can stop the node by pressing Ctrl-C on the terminal
where roslaunch was executed. In this case, you can use Ctrl-C in the
terminal where you launched the camera.launch.
You should see the red light on the camera turn off in a few seconds.
Note that the joystick.launch is still up and running, so you can still drive
the vehicle with the joystick.
Andrea
So far, we have been spelling out all commands for you, to make sure that you understand what is going on.
Now, we will tell you about some shortcuts that you can use to save some time.
in the future you will have to debug problems, and these problems might be harder to understand if you rely blindly on the shortcuts.
Time: 5 minutes.
You will know about some useful shortcuts.
Instead of using
$ ssh username@robot name.local
You can set up SSH so that you can use:
$ ssh my-robot
To do this, create a host section in ~/.ssh/config on your laptop with the following
contents:
Host my-robot
User username
Hostname robot name.local
Here, you can choose any other string in place of βmy-robotβ.
Note that you cannot do
$ ping my-robot
You havenβt created another hostname, just an alias for SSH.
However, you can use the alias with all the tools that rely
on SSH, including rsync and scp.
set_ros_master.shInstead of using:
$ export ROS_MASTER_URI=http://robot name.local:11311/
You can use the βset_ros_master.shβ script in the repo:
$ source set_ros_master.sh robot name
Note that you need to use source; without that, it will not work.
Andrea Daniele
You can run the joystick demo remotely. The procedure is documented in Unit C-13 - RC control launched remotely.
Calibrate the wheels of the Duckiebot such that it goes in a straight line when you command it to. Set the maximum speed of the Duckiebot.
The motors used on the Duckiebots are called βVoltage-controlled motorsβ. This means that the velocity of each motor is directly proportional to the voltage it is subject to. Even though we use the same model of motor for left and right wheel, they are not exactly the same. In particular, every motor responds to a given voltage signal in a slightly different way. Similarly, the wheels that we are using look βidenticalβ, but they might be slightly different.
If you drive the Duckiebot around using the joystick, you might notice that it doesnβt really go in a straight line when you command it to. This is due to those small differences between the motors and the wheels explained above. Different motors can cause the left wheel and right wheel to travel at different speed even though the motors received the same command signal. Similarly, different wheels travel different distances even though the motors made the same rotation.
It might be helpful to talk about the ROS Parameter Server here, or at least reference another page. -AD
We can counter this behavior by calibrating the wheels. A calibrated Duckiebot sends two different signals to left and right motor such that the robot moves in a straight line when you command it to.
The relationship between linear and angular velocity of the robot and the velocities of left and right motors are:
\begin{align*} v_{\text{right}} &= (g + r) \cdot (v + \dfrac{1}{2} \omega l ) \\ v_{\text{left}} &= (g - r) \cdot (v - \dfrac{1}{2} \omega l ) \end{align*}
where $v_{\text{right}}$ and $V_{\text{left}}$ are the velocities of the two motors, $g$ is called gain, $r$ is called trim, $v$ and $\omega$ are the desired linear and the angular velocity of the robot, and $l$ is the distance between the two wheels. The gain parameter $g$ controls the maximum speed of the robot. With $g \gt{} 1.0$, the vehicle goes faster given the same velocity command, and for $g \lt{} 1.0$ it goes slower. The trim parameter $r$ controls the balance between the two motors. With $r \gt{} 0$, the right wheel will turn slightly more than the left wheel given the same velocity command; with $r \lt{} 0$, the left wheel will turn slightly more the right wheel.
It might be helpful to add the differential equations that relate velocities and voltages of the motors. -AD
trim parameterThe trim parameter is set to $0.00$ by default, under the assumption that both motors and wheels are perfectly identical. You can change the value of the trim parameter by running the command:
duckiebot $ rosservice call /robot name/inverse_kinematics_node/set_trim -- trim value
Calibrate the trim parameter using the following steps.
Make sure that your Duckiebot is ON and connected to the network.
On your Duckiebot, launch the joystick process:
duckiebot $ roslaunch duckietown joystick.launch veh:=robot name
Use some tape to create a straight line on the floor (Figure 16.1).
Place your Duckiebot on one end of the tape. Make sure that the Duckiebot is perfectly centered with respect to the line.
Command your Duckiebot to go straight for about 2 meters. Observe the Duckiebot from the point where it started moving and annotate on which side of the tape the Duckiebot drifted (Figure 16.2).
Measure the distance between the center of the tape and the center of the axle of the Duckiebot after it traveled for about 2 meters (Figure 16.3).
Make sure that the ruler is orthogonal to the tape.
If the Duckiebot drifted by less than $10$ centimeters you can stop calibrating the trim parameter. A drift of $10$ centimeters in a $2$ meters run is good enough for Duckietown. If the Duckiebot drifted by more than $10$ centimeters, continue with the next step.
If the Duckiebot drifted to the left side of the tape, decrease the value of $r$, by running, for example:
duckiebot $ rosservice call /robot name/inverse_kinematics_node/set_trim -- -0.1
If the Duckiebot drifted to the right side of the tape, increase the value of $r$, by running, for example:
duckiebot $ rosservice call /robot name/inverse_kinematics_node/set_trim -- 0.1
Repeat the steps 4-8.
gain parameterThe gain parameter is set to $1.00$ by default. You can change its value by running the command:
duckiebot $ rosservice call /robot name/inverse_kinematics_node/set_gain -- gain value
You wonβt really know if itβs right until you verify it though! onto the next section
Construct a calibration station similar to the one in Figure 16.4:
The following are the specs for this 3x1 mat βrunwayβ:
Red line as close to the edge without crossing the interlocking bits
Blue/Black line 8 cm from red line and parallel to it.
White lines on the edge without intersecting the interlocking bits
Yellow line in the middle of the white lines
Blue/black start position is ~3-4 cm from the edge (not including the interlocking bits)
Place your robot as shown in Figure 16.4.
On your robot execute:
duckiebot $ cd Duckietown root
duckiebot $ make hw-test-kinematics
You should see your robot drive down the lane. If it is calibrated properly, you will see a message saying that it has PASSED, otherwise it is FAILED and you should adjust your gains based on what you observe and try again.
When you are all done, save the parameters by running:
duckiebot $ rosservice call /robot name/inverse_kinematics_node/save_calibration
The first time you save the parameters, this command will create the file
DUCKIEFLEET_ROOT/calibrations/kinematics/robot name.yaml
You can add and commit it to the repository. Then you should create a pull request in the duckiefleet repo
You can see the camera image on the laptop. The procedure is documented in Unit C-14 - RC+camera remotely.
You have all the repositories (described in Unit M-7 - Git usage guide for Fall 2017) cloned properly and you have your environment variables set properly.
Calibration for the robot camera.
Download and print a PDF of the calibration checkerboard (A4 intrinsic, A3 extrinsic, US Letter). Fix the checkerboard to a planar surface.
the squares must have side equal to 0.031 m = 3.1 cm.
Make sure your Duckiebot is on, and both your laptop and Duckiebot are connected to the duckietown network.
Open two terminals on the laptop.
In the first terminal, log in into your robot using SSH and launch the camera process:
duckiebot $ cd duckietown root
duckiebot $ source environment.sh
duckiebot $ roslaunch duckietown camera.launch veh:=robot name raw:=true
In the second laptop terminal run the camera calibration:
laptop $ cd duckietown root
laptop $ source environment.sh
laptop $ source set_ros_master.sh robot name
laptop $ roslaunch duckietown intrinsic_calibration.launch veh:=robot name
You should see a display screen open on the laptop (Figure 17.2).
Position the checkerboard in front of the camera until you see colored lines overlaying the checkerboard. You will only see the colored lines if the entire checkerboard is within the field of view of the camera.
You should also see colored bars in the sidebar of the display window. These bars indicate the current range of the checkerboard in the cameraβs field of view:
Also, make sure to focus the image by rotating the mechanical focus ring on the lens of the camera.
Do not change the focus during or after the calibration, otherwise your calibration is no longer valid. Iβd also suggest to not to use the lens cover anymore; removing the lens cover changes the focus. -MK
Now move the checkerboard right/left, up/down, and tilt the checkerboard through various angles of relative to the image plane. After each movement, make sure to pause long enough for the checkerboard to become highlighted. Once you have collected enough data, all four indicator bars will turn green. Press the βCALIBRATEβ button in the sidebar.
Calibration may take a few moments. Note that the screen may dim. Donβt worry, the calibration is working.
If you are satisfied with the calibration, you can save the results by pressing the βCOMMITβ button in the side bar. (You never need to click the βSAVEβ button.)
This will automatically save the calibration results on your Duckiebot:
duckiefleet root/calibrations/camera_intrinsic/robot name.yaml
Now letβs push the robot name.yaml file to the git repository. You can stop the camera.launch terminal with Ctrl-C or open a new terminal in Byobu with F2.
Update your local git repository:
duckiebot $ cd duckiefleet root
duckiebot $ git pull
duckiebot $ git status
You should see that your new calibration file is uncommitted. You need to commit the file to your branch.
duckiebot $ git checkout -b Github username-devel
duckiebot $ git add calibrations/camera_intrinsic/robot name.yaml
duckiebot $ git commit -m "add robot name intrinsic calibration file"
duckiebot $ git push origin Github username-devel
Before moving on to the extrinsic calibration, make sure to kill all running processes by pressing Ctrl-C in each of the terminal windows.
Arrange the Duckiebot and checkerboard according to Figure 17.5. Note that the axis of the wheels should be aligned with the y-axis (Figure 17.5).
Please re-do the picture above with a conforming Duckiebot. A conforming Duckiebot has a duckie on top. -AC
Figure 17.6 shows a view of the calibration checkerboard from the Duckiebot. To ensure proper calibration there should be no clutter in the background and two A4 papers should be aligned next to each other.
Log in into your robot using SSH and launch the camera:
duckiebot $ cd duckietown root
duckiebot $ source environment.sh
duckiebot $ roslaunch duckietown camera.launch veh:=robot name raw:=true
Run the ground_projection_node.py node on your laptop:
laptop $ cd duckietown root
laptop $ source environment.sh
laptop $ source set_ros_master.sh robot name
laptop $ roslaunch ground_projection ground_projection.launch veh:=robot name local:=true
Check that everything is working properly. In a new terminal (with environment sourced and ros_master set to point to your robot as above)
laptop $ rostopic list
You should see new ros topics:
/robot name/camera_node/camera_info
/robot name/camera_node/framerate_high_switch
/robot name/camera_node/image/compressed
/robot name/camera_node/image/raw
/robot name/camera_node/raw_camera_info
The ground_projection node has two services. They are not used during operation. They just provide a command line interface to trigger the extrinsic calibration (and for debugging).
laptop $ rosservice list
You should see something like this:
...
/robot name/ground_projection/estimate_homography
/robot name/ground_projection/get_ground_coordinate
...
If you want to check whether your camera output is similar to the one at the Figure 17.6 you can start rqt_image_view:
laptop $ rosrun rqt_image_view rqt_image_view
In the rqt_image_view interface, click on the drop-down list and choose the image topic:
/robot name/camera_node/image/compressed
Now you can estimate the homography by executing the following command (in a new terminal):
laptop $ rosservice call /robot name/ground_projection/estimate_homography
This will do the extrinsic calibration and automatically save the file to your laptop:
duckiefleet root/calibrations/camera_extrinsic/robot name.yaml
As before, add this file to your local Git repository on your laptop, push the changes to your branch and do a pull request to master. Finally, you will want to update the local repository on your Duckiebot.
it is recommended that you log to your USB and not to your SD card.
To mount your USB see Unit J-15 - Mounting USB drives.
For example, if you want to drive the robot around and collect image data you could run:
duckiebot $ make demo-joystick-camera
But anything could do.
Run on the laptop:
laptop $ cd Duckietown root
laptop $ source environment.sh
laptop $ source set_ros_master.sh robot name
laptop $ rqt_image_view
and verify that indeed your camera is streaming imagery.
To log everything that is being published, on the Duckiebot in a new terminal (See Unit J-26 - Byobu):
duckiebot $ make log-full
where here we are assuming that you are logging to the USB and have followed Unit J-15 - Mounting USB drives.
To log only the imagery, camera_info, the control commands and a few other essential things, on the Duckiebot in a new terminal (See Unit J-26 - Byobu):
duckiebot $ make log-minimal
where here we are assuming that you are logging to the USB and have followed Unit J-15 - Mounting USB drives.
On the Duckiebot run:
duckiebot $ rosbag info FULL_PATH_TO_BAG --freq
Then:
verify that the βdurationβ of the log seems βreasonableβ - itβs about as long as you ran the log command for
verify that the βsizeβ of the log seems βreasonableβ - the log size should grow at about 220MB/min
verify in the output that your camera was publishing very close to 30.0Hz and verify that you joysick was publishing at a rate between 3Hz and 6Hz.
More complex log verification methods.
The Raspberry Pi of the Duckiebot is connected to the battery.
The Stepper Motor HAT is connected to the battery.
You have a problem!
The red LED on the Raspberry Pi is OFF
Press the button on the side of the battery (Figure 19.1).
When I run ssh robot_name.local I get the error ssh: Could not resolve hostname robot_name.local.
Make sure that your Duckiebot is ON. Connect it to a monitor, a mouse and a keyboard. Run the command
duckiebot $ sudo service avahi-daemon status
You should get something like the following
β avahi-daemon.service - Avahi mDNS/DNS-SD Stack
Loaded: loaded (/lib/systemd/system/avahi-daemon.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2017-10-22 00:07:53 CEST; 1 day 3h ago
Main PID: 699 (avahi-daemon)
Status: "avahi-daemon 0.6.32-rc starting up."
CGroup: /system.slice/avahi-daemon.service
ββ699 avahi-daemon: running [robot_name_in_avahi.local
ββ727 avahi-daemon: chroot helpe
Avahi is the module that in Ubuntu implements the mDNS responder. The mDNS responder is responsible for advertising the hostname of the Duckiebot on the network so that everybody
else within the same network can run the command ping robot_name.local and reach your Duckiebot. Focus on the line containing the hostname published by the avahi-daemon on the network (i.e., the line that contains robot_name_in_avahi.local).
If robot_name_in_avahi matches the robot_name, go to the next Resolution point.
If robot_name_in_avahi has the form robot_name-XX, where XX can be any number,
modify the file /etc/avahi/avahi-daemon.conf as shown below.
Identify the line
use-ipv6=yes
and change it to
use-ipv6=no
Identify the line
#publish-aaaa-on-ipv4=yes
and change it to
publish-aaaa-on-ipv4=no
Restart Avahi by running the command
duckiebot $ sudo service avahi-daemon restart
I can SSH into my Duckiebot and run the joystick demo but the joystick does not move the wheels.
Press the button on the side of the battery (Figure 19.1).
Check that the red indicator on the joystick stopped blinking.
The joystick is connected (as shown in Figure 19.2b - Bad joystick status) but the Duckiebot still does not move.
Make sure that the controller is connected to the Duckiebot and that the OS receives the data from it. Run
duckiebot $ jstest /dev/input/js0
If you receive the error
jstest: No such file or directory
it means that the USB receiver is not connected to the Raspberry Pi or is broken. If the command above shows something like the following
Driver version is 2.1.0.
Joystick (ShanWan PC/PS3/Android) has 8 axes (X, Y, Z, Rz, Gas, Brake, Hat0X, Hat0Y)
and 15 buttons (BtnX, BtnY, BtnZ, BtnTL, BtnTR, BtnTL2, BtnTR2, BtnSelect, BtnStart, BtnMode, BtnThumbL, BtnThumbR, ?, ?, ?).
Testing ... (interrupt to exit)
Axes: 0: 0 1: 0 2: 0 3: 0 4:-32767 5:-32767 6: 0 7: 0 Buttons: 0:off 1:off 2:off 3:off 4:off 5:off 6:off 7:off 8:off 9:off 10:off 11:off 12:off 13:off 14:off
it means that the USB receiver is connected to the Raspberry Pi. Leave the terminal above open and use the joystick to command the Duckiebot. If you observe that the numbers shown in the terminal change according to the commands sent through the joystick than the problem is in ROS. Make sure that the joystick demo is launched. Restart the Duckiebot if needed and try again.
If the numbers do not change while using the joystick then follow this guide at the next Resolution point.
The controller might be connected to another Duckiebot nearby. Turn off the controller, go to a room with no other Duckiebots around and turn the controller back on. Retry.
Duckietowns are the cities where Duckiebots drive. Here, we provide a link to all bits and pieces that are needed to build a Duckietown, along with their price tag. Note that while the topography of the map is highly customable, we recommend using the components listed below. Before purchasing components for a Duckietown, read Unit D-2 - Duckietown Appearance Specification to understand how Duckietowns are built.
In general, keep in mind that:
Cost (per $m^2$): USD ??? + Shipping Fees
Time: ??? days (average shipping time)
A kit of parts ready to be assembled in a Duckietown.
Figure out costs
| Duckies | USD 17/100 pieces |
| Floor Mats | USD 37.5/6 pieces (24 sqft) |
| Duct tape - Red | USD 8.50/roll |
| Duct tape - White | USD 8.50/roll |
| Duct tape - Yellow | USD 8/roll |
| Traffic signs | USD 18.50/13 pieces |
| Total for $\text{Duckietown}/m^2$ | USD ?? |
Add suggestions for βsmallβ, βmediumβ, βbigβ towns as a function of $m^2$ and supported bots
Duckies (Figure 1.1) are essential yet non functional.
The floor mats (Figure 1.2) are the ground on which the Duckiebots drive.
We choose these mats because they have desirable surface properties, are modular, and have the right size to be street segments. Each square is (~61x61cm) and can connect on every side of other squares. There are 6 mats in each package.
Each mat can be a segment of road: straight, a curve, or an intersection (3, or 4 way). To design your Duckietown, see Unit D-2 - Duckietown Appearance Specification.
We use duck (duct) tape of different colors (Figure 1.3) for defining the roads and their signals. White indicates the road boundaries, yellow determines lane boundaries and red are stop signs.
The white and red tape we use are 2 inches wide, while the yellow one is 1 inch wide.
To verify how much tape you need for each road segment type, see Unit D-2 - Duckietown Appearance Specification.
Traffic signs (Figure 1.4) inform Duckiebots on the map of Duckietown, allowing them to make driving decisions.
Depending on the chosen road topograhy, the number of necessary road signal will vary. To design your Duckietown, see Unit D-2 - Duckietown Appearance Specification. You will also need to print out and the signs that you will need as described in Unit D-3 - Signage.
Traffic lights regulate intersections in Duckietown. Here, we provide a link to all bits and pieces that are needed to build a traffic light, along with their price tag. You will need one traffic per either three, or four way intersections. The components listed below meet the appearence specifications described in Unit D-2 - Duckietown Appearance Specification.
In general, keep in mind that:
Cost: USD ??? + Shipping Fees
Time: ??? days (average shipping time)
A kit of parts ready to be assembled in a traffic light.
Estimate time and costs
| Raspberry Pi | USD ?? |
| 4 LEDs | USD ?? |
| Wires | USD ?? |
| Total for Traffic Light | USD ?? |
Complete table
(Figure 1.5) are essential yet non functional.
Liam
This document describes the Duckietown specification. Any Duckietown not adhering to the rules described here cannot call itself a βDuckietownβ, since it is not one. Additionally, any Duckietown not adhering to these rules may cause the Duckiebots to fail in unexpected ways. These are a set of rules for which a functional system has been verified.
Note here the changes to the specification, so that we are able to keep in sync the different Duckietowns.
Version 1.0 - used for MIT 2.166
Version 2.0 - current version
Duckietown is built with two layers:
the visual appearance of the area where the Duckietown is created is variable. If you discover that this appearance is causing negative performance, a βwallβ of blank tiles constructed vertically can be used to reduce visual clutter.
Each tile is a 2 ft x 2 ft square and is able to interlock with the others.
There are five primary types of tiles, as shown in Figure 2.1
There are 3 colors of tapes: white, yellow, and red.
A Duckiebot never collides with Duckietown if it never crosses or touches a white tape strip.
Here are some facts about the white tapes:
White tapes must be solid (not dashed)
The width of the white tape is 1 inch.
The white tape is always placed on the right hand side of a lane. We assume that the Duckiebots drive on the right hand side of the road.
this should be part of the βtraffic rulesβ sections.
On a two-way road, the yellow tape should be dashed. Each piece should have a length of approximately 2 in with a 1 in gap separating each piece.
Yellow tapes on curves: see curved road image in white tape section, pieces at tile edges should be in center of lane, piece at the middle of the curve should be approximately 20.5 cm from middle of inner center white piece of tape, with approximated circular arc in between.
Red tapes MAY only appear on intersection tiles.
The red tape must be the full width of the duck tape roll and should cross the entire lane perpendicular to the lane.
what is the width of the roll? 1 in? - AC
Red and White rolls have a 2 inch thickness. Yellow is 1 inch wide. -JT
The placement of red tape should always be under yellow and white tape.
A Duckiebot navigates Duckietown by a sequence of:
The guarantee is:
If the Duckiebot stops before or ON the red strip, no collisions are possible.
Here are some topological rule constraints that must be met:
An intersection can NOT be adjacent to a curved road tile or another intersection tile.
Any two adjacent non-empty tiles must have a feasible path from one to the other of length two: if they are adjacent, they must be connected.
Some examples of non-conforming topologies are shown in Figure 2.3.
An experimental new development.
A parking lot is a place for Duckiebots to go when they are tired and need a rest.
A parking lot introduces three additional tile types:
the tape on the spot and spot access tiles is currently not yet specified.
The following are the rules for a conforming parking lot:
this point needs further specification
Experimental
A βlaunch tileβ is used to introduce a new Duckiebot into Duckietown in a controllable way. The launch file should be placed adjacent to a turn tile so that a Duckiebot may βmergeβ into Duckietown once the initialization procedure is complete.
Specification for tape on the launch tile
A βyieldβ sign should be visible from the launch tile.
IMPORTANT: All signage should sit with base on the floor and stem coming through the connection between the tiles. Generally, it is advisable to adhere the sign to the floor with double-sided tape. Under no circumstances should the white (any other tape) be obscured.
To print and assemble the signs refer to Unit D-3 - Signage.
Center of signs are 13 cm height with apriltags of 6.5 cm sq. and a white border pasted below them.
The allowable traffic signs are as in Figure 2.4.
Signs may appear on the opposite side and at the corner of the adjacent tile from which they are viewed. In the absence of any signs, it is assumed that all network flows are allowed so a sign MUST be placed and visible whenever this is not the case.
Signs must only be placed on empty tiles, or next to one of the other tile types if on the border of a map. The sign placements for four different cases are shown in Figure 2.5. At intersections, from each stop line 2 signs should be clearly visible: 1) the intersection type (traffic light or stop sign) and 2) the intersection topology.
At present, 4-way intersections must be equipped with traffic lights for safe navigation.
On straight and curved roads, additional signs can be added as desired. Their placement is indicated in Figure 2.5c - straight road and Figure 2.5d - curved road. The signs should be placed at the border between two tiles and should face towards oncoming traffic as indicated.
In these figures the arrow is the direction of the sign.
Font: arial.
Color: Perhaps we could start with real-world settings: white as foreground and green as background.
Border: currently no additional borders
The rounded corners are modified into 90 degrees.
Height: sign board height is 1.5 in. (2.1 in),
Width: Currently 4.5 in for id 500-511. (6.1 in +1.1 in βSTβ or 5.5 in + 1.7 in βAVEβ)
Alphabet = English upper case. Different writing systems may need different algorithms.
Text direction: Horizontal for alphabetical languages
Every segment of road must have at least one road name sign.
Every turn tile should have a road name sign.
The placement of the road name signs is as indicated in Figure 2.6.
Street name signs should never be perpendicular to the road - they are too big and obtrusive.
The assembly procedure for building the a traffic light is found in Unit D-5 - Traffic lights Assembly
towrite
The lights must be at a height of exactly 20 cm above the center of the intersection tile.
The Raspberry Pi should sit on a pole that is based at the corner of the tile outside of the allowable driving region.
Liam
The raw materials as described in Unit D-1 - Duckietown parts
A set of signs to be used for assembling your Duckietown.
Before beginning with sign assembly you should design a map that adheres to the specification.
An example that was used for the 2017 version of the class is here: Unit M-30 - The Map Description
The full set of currently existing signs is available here: pdf docx
The set of tags used for the 2017 map are available here: pdf docx
If you find that what is available in the database in insufficient for your needs, then you will need to add to the existing database.
To do so you will have to load the original AprilTags file available here: pdf ps
Which tag you should use depends on what type of sign you are trying add. The ranges of tags are specified in Table 3.1.
| Purpose | Size | Family | ID Range |
| Traffic signs | 6.5cm x 6.5cm | 36h11 | 1-199 |
| Traffic lights | 6.5cm x 6.5cm | 36h11 | 200-299 |
| Localization | 6.5cm x 6.5cm | 36h11 | 300-399 |
| Street Name Signs | 6.5cm x 6.5cm | 36h11 | 400-587 |
First, find the last sign of the type that you are trying to make in the signs and tags doc. You will use the next available ID after this one.
Construct the new sign by first copying and pasting an existing sign of similar type, and then replacing/adding the new AprilTag. To add the new april tag, use a screen capture mathod to crop precisely around the tag at the top and sides and include the sign id at the bottom. Then paste the tag into your word file under your desired and resize it exactly 6.5cm (2.56inches).
If you make a new road name sign, you may need to change the font size of the name so that it appears on one line (this is why we like people with names like βROYβ and βRUSβ).
Important: You must also add your new sign to the April Tags DB in the software repo.
Add a new block like the ones that already exists or modify the one with the appropriate tag id:
- tag_id: NEW_TAG_ID
tag_type: in {TrafficSign, Light, Localization, StreetName}
street_name: either NEW_STREET_NAME or blank
vehicle_name: currently not used
traffic_sign_type: either TRAFFIC_SIGN_TYPE or blank
The value of NEW_STREET_NAME is up to you to decide (have fun with it!). The value of TRAFFIC_SIGN_TYPE should be one of the signs in Figure 2.4
When finished, regenerate the PDF version of the Word file, and commit everything to the repo (via a pull request of course).
It is also possible of course to start you own completely different signs and tags database, but make sure that you specify in the april_tags code which database to load from.
Update the way that the april tags code loads the database
To assemble the signs, you should print out the pdf version of the signs and tags file on the thickest card stock available. Cut the signs out with a straight edge and a very sharp knife, leaving a small border of white around the sign. Then use double-sided tape or some other adhesive to affix the the paper sign to the wooden base.
For placement of signs see Subsection 2.5.3 - Placement.
Shiying
Follow the rules in the Unit D-2 - Duckietown Appearance Specification.
do we need this page at all?
Marco Erni
???
headlights: white, constant
Assumption:
20 fps to do LED detection
1s to decide
3 frequencies to detect
tail lights: red, 6 hz square wave
traffic light βGOβ = green, 1 hz square wave
traffic light βSTOPβ = red, 1.5 Hz square wave
duckie light on top, state 0 = off
duckie light on top, state 1 = blue, 3 Hz, square wave
duckie light on top, state 2 = ?, 2.5 Hz square wave
duckie light on top, state 3 = ?, 2 Hz square wave
DB17-lc configurationsThis section contains the acquisition, assembly and setup instructions for the DB17-lc configurations. These instructions are separate from the rest as they approximately match the b releases of the Fall 2017 Duckietown Engineering Co. branches.
To understand how configurations and releases are defined, refer to: Unit C-1 - Duckiebot configurations.
DB17-lc)Upgrading your DB17 (DB17-wjd) configuration to DB17-lc (DB17-wjdlc) starts here, with purchasing the necessary components. We provide a link to all bits and pieces that are needed to build a DB17-lc Duckiebot, along with their price tag. If you are wondering what is the difference between different Duckiebot configurations, read Unit C-1 - Duckiebot configurations.
In general, keep in mind that:
A Duckiebot in DB17-wjd configuration.
Cost: USD 77 + Bumpers manufacturing solution
Time: 21 Days (LED board manufacturing and shipping time)
A kit of parts ready to be assembled in a DB17-lc configuration Duckiebot.
LEDs (DB17-l) |
USD 10 |
LED HAT (DB17-l) |
USD 28.20 for 3 pieces |
Power Cable (DB17-l) |
USD 7.80 |
20 Female-Female Jumper Wires (300mm) (DB17-l) |
USD 8 |
Male-Male Jumper Wire (150mm) (DB17-l) |
USD 1.95 |
PWM/Servo HAT (DB17-l) |
USD 17.50 |
| Bumpers | TBD (custom made) |
40 pin female header (DB17-l) |
USD 1.50 |
5 4 pin female header (DB17-l) |
USD 0.60/piece |
2 16 pin male header (DB17-l) |
USD 0.61/piece |
12 pin male header (DB17-l) |
USD 0.48/piece |
3 pin male header (DB17-l) |
USD 0.10/piece |
2 pin female shunt jumper (DB17-l) |
USD 2/piece |
5 200 Ohm resistors (DB17-l) |
USD 0.10/piece |
10 130 Ohm resistors (DB17-l) |
USD 0.10/piece |
Caster (DB17-c) |
USD 6.55/4 pieces |
4 Standoffs (M3.5 12mm F-F) (DB17-c) |
USD 0.63/piece |
8 Screws (M3.5x8mm) (DB17-c) |
USD 4.58/100 pieces |
8 Split washer lock (DB17-c) |
USD 1.59/100 pieces |
Total for DB17-wjd configuration |
USD 212 |
Total for DB17-lc components |
USD 77 + Bumpers |
Total for DB17-wjdlc configuration |
USD 299+Bumpers |
The Duckiebot is equipped with 5 RGB LEDs (Figure 1.1). LEDs can be used to signal to other Duckiebots, or just make fancy patterns.
The pack of LEDs linked in the table above holds 10 LEDs, enough for two Duckiebots.
The LED HAT (Figure 1.2) provides an interface for our RGB LEDs and the computational stack. This board is a daughterboard for the Adafruit 16-Channel PWM/Servo HAT, and enables connection with additional gadgets such as ADS1015 12 Bit 4 Channel ADC, Monochrome 128x32 I2C OLED graphic display, and Adafruit 9-DOF IMU Breakout - L3GD20H+LSM303. This item will require soldering.
This board is custom degined and can only be ordered in minimum runs of 3 pieces. The price scales down quickly with quantity, and lead times may be significant, so it is better to buy these boards in bulk.
The PWM/Servo HAT (Figure 1.3) mates to the LED HAT and provides the signals to control the LEDs, without taking computational resources away from the Rasperry Pi itself. This item will require soldering.
To power the PWM/Servo HAT from the battery, we use a short (30cm) angled male USB-A to 5.5/2.1mm DC power jack cable (Figure 1.4).
The Duckiebot needs one male-male jumper wire (Figure 1.5) to power the DC Stepper Motor HAT from the PWM/Servo HAT.
20 Female-Female Jumper Wires (Figure 1.6) are necessary to connect 5 LEDs to the LED HAT.
These bumpers are designed to keep the LEDs in place and are therefore used only in configuration DB17-l. They are custom designed parts, so they must be produced and cannot be bought. We used laser cutting facilities.
Upgrading DB17 to DB17-l requires several electrical bits: 5 of 4 pin female header, 2 of 16 pin male headers, 1 of 12 pin male header, 1 of 3 pin male header, 1 of 2 pin female shunt jumper, 5 of 200 Ohm resistors and finally 10 of 130 Ohm resistors.
These items require soldering.
DB17-c)The caster (Figure 1.10) is an DB17-c component that substitutes the steel omnidirectional wheel that comes in the Magician Chassis package. Although the caster is not essential, it provides smoother operations and overall enhanced Duckiebot performance.
To assemble the caster at the right height we will need to purchase:
missing figures, update caster bits figures
DB17-l)General rule in soldering
Shiying
Duckiebot DB17-l parts. The acquisition process is explained in Unit E-1 - Acquiring the parts (DB17-lc).
The configurations are described in Unit C-1 - Duckiebot configurations.
Time: 30 minutes
A DB17-l Duckiebot
General rule in soldering:
(alternative instructions: how to solder on the PWM/Servo HAT)
Put the following components on the table according the Figure
add LSD board image, top and bottom.


Put the following components according the figure on the table:
Tips:
Solder all female headers to the bottom of the board. Alignment becomes easy if the female headers are plugged into the PWM heat, and the LSD board rests on top.
Solder all male headers to the top of the board. Male header positions are outlined on the silkscreen.


Parts list:
Instructions:
Connect LED accordingly to silkscreen indication on PRi 2 LSD board
silkscreen legend: Rx, Gx, Bx are red, green, and blue channels, accordingly, where x is the LED number; C is a common line (either common anode or common cathode)
For adafruit LEDs are common anode type. The longest pin is common anode. Single pin on the side of common is red channel. The two other pins are Green and Blue channels, with the blue furthest from the common pin.
Both types of LEDs are supported. Use shunt jumper to select either common anode (CA) or common cathode (CC) on 3-pin male header. Note, however, that all LEDs on the board must be of the same type.
Stack the boards
Screw the first eight standoffs into the Pi - provide hints on the location of standoffs and the suggested orientation of the boards w/r to the chassis
connect the camera to the Pi [image showing the connector ?]
Stack the DC/Stepper Motor HAT onto the Pi, aligning both sets of GPIO pins over each other and screw the standoffs to secure it. Try to not bend the camera connector too much during this step
Stack the 16-channel PWM/Servo HAT onto the Pi, both sets of GPIO pins over each other and screw the standoffs to secure it
Slide the battery between the two chassis plates
Power the PWM/Servo HAT and Pi connecting them to the battery with the cables included in the duckiebox
Power the DC/Stepper motor from the PWM/Servo HAT using the male-to-male cable in the duckiebox, connect the positive
connect the Pi to the board
Finished!
DB17-lc)Shiying
Duckiebot DB17-lc parts.
The acquisition process is explained in Unit E-1 - Acquiring the parts (DB17-lc).
Soldering DB17-lc parts.
The soldering process is explained in Unit E-2 - Soldering boards (DB17-l).
Having assembled the Duckiebot in configuration DB17 (or any DB17-wjd). The assembly process is explained in Unit C-5 - Assembling the Duckiebot (DB17-jwd).
Time: about 30 minutes.
An assembled Duckiebot in configuration DB17-wjdlc.
DB17-l1)Recommend: If you have Bumpers or caster to assembly, it is recommend to have Bumpers and caster assembled before the PWM hat. The assembly process is explained in Unit E-4 - Bumper Assembly and [] (#caster_wheel_instruction).
From now on, the DC Motor Hat will be powered by the PWM HAT via male -male jumper wire. Before that, the previous hand-made USB power cable needed to be removed. Insert the male-male jumper wire into + power terminal on the DC motor HAT (DC-end).
Put a soldered Servo/PWM HAT board (in your Duckiebox) with 4 standoffs on the top of Stepper Motor HAT.
Insert the other end of male-male jumper wire into β+5βV power terminal on the PWM HAT (PWM-end). It leads the power to DC motor HAT.
To power the PWM/Servo HAT from the battery, plugin a short (30cm) angled male USB-A to 5.5/2.1mm DC power jack cable into PWM HAT. The other end of the power cable will plugin to the battery when it is in use.
DB17-l2)For instructions on how to assemble your bumpers set, refer to: Unit E-4 - Bumper Assembly.
DB17-l3)For instructions on how to assemble the LED HAT and related LEDs, refer to: Unit E-5 - DB17-l setup.
finish above, estimate assembly time, add bumper assembly instructions, add LED positioning and wiring instructions, add castor wheel assembly instructions
Duckiebot DB17-lc parts.
Having the Duckiebot with configuration DB17-wjd assembled. The assembly process is explained in Unit E-3 - Assembling the Duckiebot (DB17-lc).
Time: about 15 minutes.
A Duckiebot with Bumpers (configuration DB17-l2)
The following should be included in your parts envelope (See image below for these components):
The following is not included in your parts envelope but will be needed for assembly:
When assembling, be sure to be gentle! Tighten screws just enough so that the parts will remain stationary. When inserting LEDs, gently work them into their holders. While the acrylic is relatively tough, it can be fractured with modest force. We donβt have many replacements (at this moment) so we may not be able to replace a broken part.
Peel protective layer off of all parts on all sides.
The backside of duckiebot before assemblying the bumpers looks as Figure 4.4:
Now remove the spacers and the (short)metall screws from the standoffs (configuration βDB17-wjdβ) and replace it with 4 M3x10 nylon screws for connecting the chassis and the bumper spacers.
M3x10 screws attaching bottom rear brace:
Back View, fully assembled:
For the ETH students, the M3*10 nylon screws for attaching the rear spacers with chassis were already distributed during the duckiebot ceremony and not included in second distribution. Please reuse them!
Carefully guide rear bumper on to rear bumper brace tabs. Ensure that the hole for charging aligns with the charging port on your battery.
Locate 4 M2.5 nylon/metall nuts and 4 M2.5*10 nylon screws. Place a nut in the wide part of the t-slot and thread a screw into the nut as shown in the following pictures. Note Use care when assembling! If you are having trouble with the nuts falling out, take a small piece of transparent tape and place it over both sides of the t-slot with the nut inside. It wonβt look as nice but it will be much easier to assemble.
Test the screws and the nuts once by screwing them together before you use it for the bumpers. It make the fllowing assembly process much easier.
For ETH 2017, the screws and nuts using for this step are M2.5*10 nylon screws (white) and M2.5 nylon or metall nuts from the envelope.
The completed rear bumper should look like this:
Congrats! your rear bumper assembly is complete!
The front side of duckiebot in βDB17-wjdβ should look like in Figure 4.12
Take 2x M2.5*10 nylon screws and 2x M2.5 nylon nuts and install them as shown in the following pictures. The first picture shows the correct holes to mount these screws (The correct position is the widest pair of 3mm holes beside the camera). The nuts should tightened on by a few threads (these are the two nuts that are not yet tightened at the top of the second picture):
Before tighten the front bumper, you should organise the wires of LEDs going through the right holes of chassis. Take a look in Figure 5.4 The center LED should be bent at a right angle in the direction that the wire is fed through the body.
Take the front bumper and carefully press the LEDs into the flexure holders. Take care that the wires are routed behind the front bumper. Also note that the front center LED wire should not be crushed between the bumper and the right spacer (you will likely fracture the bumper if you try to force it). The center LED should be bent at a right angle in the direction that the wire is fed through the body (see Figure 4.13).
Position the bumper so that the nuts align with the t-slots. You may need to loosen or tigthen the screws to align the nuts. You may also need to angle the front bumper when inserting to get it past the camera screws.
Gently tighten the nuts. The front bumper should now stay in position.
Congrats! your bumper assembly is complete!
DB17-l setupShiying
Duckiebot DB17-lc parts.
The acquisition process is explained in Unit E-1 - Acquiring the parts (DB17-lc).
Soldering DB17-lc parts.
The soldering process is explained in Unit E-2 - Soldering boards (DB17-l).
Having assembled PWM Hat on the Duckiebot with configuration DB17-wjd. The assembly process is explained in Unit E-3 - Assembling the Duckiebot (DB17-lc).
Time: about 15 minutes.
A Duckiebot with LEDs attached (configuration DB17-l3)
To attach the LEDs on the duckiebots, the following components are needed:
The LEDs are common anode type. The longest pin is called the common. The single pin on the side of common is red channel. The two other pins are Green and Blue channels, with the blue furthest from the common pin.
Use the long wires with two female ends. Attach one to each of the pins on the LED.
To figure out the order to connect them to the LSD hat, use the legend on the silkscreen and the information above. i.e. RX - means the red pin, CX - means the common, GX means the green, and BX means the blue. The βXβ varies in number from 1-5 depending on which LED is being connected as discussed in the next section.
Use Tape to keep the LEDs stick with the wires.
Silkscreen legend: Rx, Gx, Bx are red, green, and blue channels, accordingly, where x is the LED number; C is a common line (either common anode or common cathode).
Define the following names for the lights:
The LEDs are wired according to Figure 5.4.
Mappings from the numbers on the LED hats to the positions shown (TOP is now the one in the middle at the front)
It is advised that the LED cables are routed through the positions noted in the images below before installing the bumpers:
Front Left, Front Middle, and Front Right LED Wiring suggestion:
Adjust the LED terminals (particularly in the front) so that they do not interfere with the wheels. This can be accomplished by bending them up, away from the treads.
This part describes a set of demos that demonstrate the functionality of the robots.
Some of these demos require different hardware on the robot, or in Duckietown.
This is the template for the description of a demo.
First, we describe what is needed, including:
Duckiebot in configuration ???
Camera calibration completed.
First, we show a video of the expected behavior (if the demo is succesful).
Here, describe the assumptions about the Duckietown, including:
Do not write instructions here. The instructions should be somewhere in the part about Duckietowns. Here, merely point to them.
Write here any special setup for the Duckiebot, if needed.
Do not write instructions here. The instructions should be somewhere in the appropriate setup part.
The pre-flight checklist describes the steps that are sufficient to ensure that the demo will be correct:
Check: operation 1 done
Check: operation 2 done
Here, give step by step instructions to reproduce the demo.
Step 1: XXX
Step 2: XXX
Add here any troubleshooting / tips and tricks required.
Finally, put here a video of how the demo can fail, when the assumptions are not respected.
???
???
to write
Theory chapters benefit from a standardized exposition. Here, we define the template for these chapters. Rememeber to check Unit B-2 - Basic Markduck guide for a comprehensive and up-to-date list of Duckiebook supported features.
Start with a brief introduction of the discussed topic, describing its place in the bigger picture, justifying the reading constraints/guidelines below. Write it as if the reader knew the relevant terminology. For example:
PID control is the simplest approach to making a system behave in a desired way rather than how it would naturally behave. It is simple because the measured output is directly feedbacked, as opposed to, e.g., the systemβs states. The control signal is obtained as a weighted sum of the tracking error (_P_roportional term), its integral over time (_I_ntegrative term) and its instantaneous derivative (_D_erivative term), from which the appellative of PID control. The tracking error is defined as the instantaneous difference between a reference and a measured system output.
Knowledge necessary:
Required Reading: Insert here a list of topics and suggested resources related to necessary knowledge in order to understand the content presented. Example:
Terminology: autonomy overview
System Modeling: basic kinematics, basic dynamics, linear algebra, State space representations, Linear Time Invariant Systems
Suggested Reading: Insert here a list of topics and suggested resources related to recommended knowledge in order to better understand the content presented. Example:
Definitions of Stability, Performances and Robustness: [7], β¦
observability/detectability and controllability/reachability: [7]
Discrete time PID: [7]
Bode diagrams: [7]
Nyquist plots: [7]
[β¦]
In this section we crisply define the problem object of this chapter. It serves as a very brief recap of exactly what is needed from previous atoms as well. E.g.
Let:
\begin{align} \dot{\state}_t = A\state_t+Bu_t \\ y = C\state_t+Du_t \label{eq:system}\tag{1} \end{align}
be the LTI model of the Duckiebotβs plant, with $x \in \statesp$, $y \in \mathbb{R}^p$ and $u \in \mathbb{R}^m$. We recall (Duckiebot Modeling) that:
\begin{align} A &= \left[ \begin{array}{ccc} a_{11} & \dots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{n1} & \dots & a_{nn} \end{array} \right] \\ B &= \left[ b_1 \,\, \dots \,\, b_m \right]^T \\ C &= \left[ c_1 \ \,\, \dots \,\, c_p \right] \\ D &= 0. \end{align}
[β¦]
Remember you can use the problem environment of $\LaTeX$ to formally state a problem:
PID Given a system \eqref{eq:system} and measurements of the output $\tilde{y}_t = y_t + n_t, n_t \sim \cal{N}(0,\sigma)$, find a set of PID coefficients that meet the specified requirements for: - stability, - performance, - robustness.
as shown in (Figure 1.1).
Reference signals A reference signal $\tilde{y}_t \in \mathcal{L}_2(\mathcal{T})$ is β¦
Definition 13 - Reference signals is very important.
Insert βrandomβ checks to keep the readerβs attention up:
if you canβt be woken up in the middle of the night and remember the definition of $\mathcal{L}_2(\cdot)$, read: [7]
Another definition Lorem
Now that we know what weβre talking about, lets get in the meat of the problem. Here is what is happening:
$$ \cal{Lorem} $$
Introduce the βsynthesis through attemptsβ methodology (a.k.a. tweak until death)
How do we know if the PID controller designed above is doing well? We need to define some performance metrics first:
Overshoot, Module at resonance, Settling Time, Rising Time
[β¦]
This is a βthink about itβ interrupt, used as attention grabber:
When a Duckiebot βovershootsβ, it means that [β¦] and the following will happen [β¦].
And finally, this is how you save the world, in theory.
This section serves as a collection of theoretical and practical examples that can clarify part or all of the above.
More academic examples
Immagine a spring-mass-damper systemβ¦
[β¦]
More Duckiebot related examples
[β¦]
Here we just add references to the suggested exercises, defined in the appropriate exercise chapters.
Strong of this new knowledge (what have we learned), we can now [β¦].
Further Reading: insert here reference resources for the interested reader:
learn all there is to know about PID: [7]
become a linear algebra master: Matrix cookbook
Do not include a reference chapter. References are automatically compiled to the Bibliography Section.
Jacopo
Jacopo
Andrea
You should not have to use presentation macros like \mathcal, \boldsymbol, etc.;
rather, for each class of things that we have (set, matrices, random variables, etc.)
we are going to define a LaTeX macro.
Use the macro \aset{X} to refer to the set $\aset{X}$.
Use the macro \amat{M} to refer to the matrix $\amat{M}$.
To indicate tuples, use the macro \tup, which produces $\tup{a,b,c}$.
If $x$ is a function of time, use $x_t$ rather than $x(t)$.
Consider the function $x(t)$.
Consider the function $x_t$.
To refer to the time variable, use \Time: $t \in \Time$.
To refer to a random variable, use the macro \rv.
This is rendered using a bold symbol.
$p(\rv{x}=x_0)$ is the probability that the random variable $\rv{x}$ has the value $x_0 \in \aset{X}$.
Use \reals for the real numbers.
Use \nats for the natural numbers.
Use \ints for the integers numbers.
| command | result | |
\aset{X}, \aset{Y} |
$\aset{X}, \aset{Y}$ | Symbols for sets |
\amat{M}, \amat{P} |
$\amat{M}, \amat{P}$ | Symbols for matrices |
\avec{u}, \avec{v} |
$\avec{u}, \avec{v}$ | Symbols for vectors |
\nats |
$\nats$ | Natural numbers |
\ints |
$\ints$ | Integers |
\reals |
$\reals$ | Real numbers |
\definedas |
$\definedas$ | Defined as |
\tup{a,b,c} |
$\tup{a,b,c}$ | Tuples |
\Time |
$\Time$ | Time axis |
Here are some useful symbols to refer to geometric spaces.
| command | result | |
\SOthree |
$\SOthree$ | Rotation matrices |
\SEthree |
$\SEthree$ | Euclidean group |
\SEtwo |
$\SEtwo$ | Euclidean group |
\setwo |
$\setwo$ | Euclidean group algebra |
States and poses:
| command | result | |
\pose |
$\pose_t \in \SEtwo$ | Pose of the robot in the plane |
\state_t \in \statesp |
$\state_t \in \statesp$ | System state (includes the pose, and everything else) |
Dzenan
k:sets
Set A set $\aset{X} = \{x_1, x_2, \dots\}$ is a well-defined collection of distinct elements, or members of the set, $x_i$, $i = 1, 2, \dots$.
k:sets
k:maps
We define a function (or map) as a mapping between sets.
Function A function $f : \aset{X} \to \aset{Y}$ is a mapping between the sets $\aset{X}$ and $\aset{Y}$. For every input element $x \in \aset{X}$, the mapping will associate an output $y = f(x) \in \aset{Y}$.
Maps can be classified by the nature of the relationship between inputs and outputs in: injective, surjective or bijective add-ref.
to write
to write
to write
k:sets
k:naturals, k:integers, k:reals
$\nats = \{0, 1, 2, \cdots\}$
The natural numbers are the set positive numbers, including zero.
Given two natural their addition is always a natural number:
$$ a+b = c \in \nats, \forall a,b \in \nats. \label{eq:intro-nats}\tag{1}$$
The same does not hold of the subtraction operation:
$$ a-b = c \in \nats \iff a \geq b. $$
For this reason set of integer numbers is defined.
$\ints = \{\cdots, -2, -1, 0, 1, 2, \cdots \}$
The integers are the set of positive and negative natural numbers, including the zero. By definition, the set of integers includes the naturals: $\ints \subset \nats$.
The sum (i.e., addition and subtraction) of two integers is always an integer: \begin{align} a + b &= c \in \ints, \forall a,b \in \nats \\ a - b &= c \in \ints, \forall a,b \in \nats. \end{align}
The multiplication of two integers is always an integer, but the same does not apply for the division operation:
$$ \frac{a}{b} = c \in \ints \iff a = kb, k \in \ints, b \neq 0. $$
For this reason the rational numbers are introduced.
The set of rational numbers includes all fractions of integers: $\rats = \{ c | \frac{a}{b} = c, a,b \in \ints, b \neq 0 \}$.
The set of rational number is complete under sum and product (i.e., multiplication and division), but not under other operations such as the root. E.g., $\sqrt{2}$ cannot be expressed as a fraction of two integers. These numbers are not rational, and therefore are defined as irrationals.
Irrational numbers are all those numbers that cannot be expressed as a fraction. Notable examples of irrational numbers include the aforementioned $\sqrt{2}$, but even pi ($\pi$) and the Euler number ($e$).
Irrational numbers are not typically referred to as as a set by themselves, rather, the union of the rational and irrational numbers defines the set of reals.
The real numbers ($\reals$) are arguably the most used set of numbers, and are often considered the default set if no specification is provided.
The real numbers are defined as the union of rational and irrational numbers, and therefore by definition include the integers and the naturals.
The reals are still not complete under all βcanonicalβ operations. In fact, there is no solution to the root (of even index) of a negative number.
For this reason, the complex numbers are introduced.
Complex numbers are defined as the sum of a real and an imaginary part:
$$ z = a + ib, a,b \in \reals, i = \sqrt{-1} $$
and can be represented on the plane of Gauss, a Cartesian plane featuring the real part of $z$, $Re(z) = a$, on the x-axis and the imaginary part, $Im(z)=b$, on the y-axis (Figure 4.1).
Complex numbers introduce the concept of phase of a number, which is related to its βorientationβ, and are invaluable for describing many natural phenomena such as electricity and applications such as signal decoders.
For more information on the algebra and properties of natural numbers:
Dzenan
The powers of $i$
\begin{align} i &= sqrt{-1} \\ i^2 &= -1 \\ i^3 &= i^2 \cdot i = -i \\ i &= i^2 \cdot i^2 = 1 \\ i^5 &= i^4 /cdot i = i \\ \vdots \end{align}
Jacopo
Linear algebra provides the set of mathematical tools to (a) study linear relationships and (b) describe linear spaces. It is a field of mathematics with important ramifications.
Linearity is an important concept because it is powerful in describing the input-output behavior of many natural phenomena (or systems). As a matter of fact, all those systems that cannot be modeled as linear, still can be approximated as linear to gain an intuition, and sometimes much more, of what is going on.
So, in a way or the other, linear algebra is a starting point for investigating the world around us, and Duckietown is no exception.
This chapter is not intended to be a comprehensive compendium of linear algebra.
this reference
this other reference
add references throughout all chapter
Real numbers are complex for you?: Number theory addref
$\forall$ is a typo for A and $\in$ are Euros? Mathematical symbolic language.
In this section we discuss vectors, matrices and linear spaces along with their properties.
Before introducing the these arguments, we need to formally define what we mean by linearity. The word linear comes from the latin linearis, which means pertaining to or resembling a line. You should recall that a line can be represented by an equation like $y = mx + q$, but here we intend linearity as a property of maps, so there is a little more to linearity than lines (although lines are linear maps indeed).
To avoid confusions, let us translate the concept of linearity in mathematical language.
Linearity A function $f: \aset{X} \to \aset{Y}$ is linear when, $\forall x_i \in \aset{X}$, $i = \{1,2\}$, and $\forall a \in \reals$:
\begin{align} f(ax_1) &= af(x_1), \label{eq:lin1}\tag{2} \quad \text{and:} \\ f(x_1 + x_2) &= f(x_1) + f(x_2) \label{eq:lin2}\tag{3} \end{align}
Condition \eqref{eq:lin1} is referred to as the property of homogeneity (of order 1), while condition \eqref{eq:lin2} is referred to as additivity.
Superposition Principle Conditions \eqref{eq:lin1} and \eqref{eq:lin2} can be merged to express the same meaning through: \begin{align} f(ax_1 + bx_2) = af(x_1) + bf(x_2), \forall x_i \in \aset{X}, i = \{1,2\}, \forall a,b \in \reals \label{eq:linearity}\tag{4}. \end{align}
This equivalent condition \eqref{eq:linearity} is instead referred to as superposition principle, which unveils the bottom line of the concept of linearity: adding up (equivalently, scaling up) inputs results in an added up (equivalently, scaled up) output.
Let $n$ belong to the set of natural numbers $\nats$, i.e., $n \in \nats$, and let $a_i \in \reals$, $i = \{1, \dots, n\}$ be real coefficients. While $\reals$ is the set of real numbers, $\reals^n$ is the set of all $n$-tuples of real numbers.
Vector and components
An $n$-dimensional $\textit{vector}$ is an $n$-tuple:
\begin{align} \label{eq:vector}\tag{5} \avec{v} = \left[ \begin{array}{c} v_1 \\ \vdots \\ v_n \end{array} \right] \in \reals^{n \times 1} \equiv \reals^n, \end{align}
of components $v_1, \dots, v_n \in \reals$.
Vector notation A more general notation for tuples can be used when denoting vectors: \begin{align} \label{eq:vector-tuple-notation}\tag{6} \avec{v} = \tup{v_1, \cdots, v_n}. \end{align} In these preliminaries, we will adopt the \eqref{eq:vector} βengineeringβ notation as it arguably simplifies remembering vector-matrix operations (Unit G-9 - Matrices and vectors).
You can imagine a vector Figure 6.1 as a βdirectional numberβ, or an arrow that starts a certain point and goes in a certain direction (in $\reals^n$). In this representation, the number is the length of the arrow, or the magnitude of the vector (sometimes referred to even as modulus), and it can be derived through the vectorβs components.
Length of a vector We define the length, or modulus, of a vector $\avec{v} \in \reals^n$ as: \begin{align} \label{eq:vec-2-norm}\tag{7} \|\avec{v}\| = \sqrt{v_1^2 + \dots + v_n^2} \in \reals. \end{align}
2-norm Generally speaking, it is not always possible to define the length of a vector (addref). But when it is possible (e.g., in Hilbert spaces), and in Duckietown it always is, there are many ways to define it. The most common and intuitive definition is the Euclidian- or 2-norm, which is defined above in \eqref{eq:vec-2-norm}.
We will discuss norms more in detail in Unit G-13 - Norms.
Unit vector A unit vector, or versor, is a vector $\avec{e}$ of of unit length: \begin{align} \label{eq:unit-vector}\tag{8} \|\avec{e}\| = 1. \end{align}
Unit vectors are used to define the directions of the components of a vector, allowing for an algebraic rather than vectorial representation. As we will see in Subsection 6.2.1 - Vector algebra, this will make the algebra of vectors more intuitive.
Let $\avec{v} \in \reals^3$ be a vector defined in the Cartesian space. Let, moreover, $(\avec{i},\avec{j},\avec{k})^T$ be the versor of the Cartesian axis, i.e.:
\begin{align}\label{eq:example-vector-algebraic}\tag{9}
\avec{i} &= [1,0,0]^T; \\
\avec{j} &= [0,1,0]^T; \\
\avec{k} &= [0,0,1]^T.
\end{align}
Then, a vector can be written equivalently in vector or algebraic form: $\avec{v} = [v_1, v_2, v_3]^T = v_1\avec{i} + v_2\avec{j}+v_3\avec{k}$. Unit vectors are sometimes explicitly denoted with a hat (^), e.g., $\hat{\avec{i}}, \hat{\avec{j}}, \hat{\avec{k}}$.
Normalizing vectors Every vector can be made into a unit vector, or normalized, by dividing each of its components by the vectorβs magnitude: \begin{align}\label{eq:vector-normalizing}\tag{10} \hat{\avec{v}} = \frac{\avec{v}}{\|\avec{v}\|} = \left[\frac{v_1}{\|\avec{v}\|}, \frac{v_2}{\|\avec{v}\|}, \frac{v_3}{\|\avec{v}\|}\right]^T. \end{align}
We here define operations amongst two given vectors defined in the same space: $\avec{u} = [u_1, u_2, u_3]^T, \avec{v} = [v_1, v_2, v_3]^T \in \reals^3$.
The sum of two vectors is a vector, and its components are the sum of the two vectors components.
Vectorial sum \begin{align} \label{eq:vector-sum}\tag{11} \avec{u} + \avec{v} = [u_1+v_1, u_2+v_2, u_3+v_3]^T. \end{align}
Sum Mathematical operations come in pairs, which represent the same concept. A sum operation, sometimes more extensively referred to as the algebric sum, is the concept of summing, i.e., it includes both addition and subtraction. (A subtraction is nothing but an addition between positive and negative numbers.)
The parallelogram law helps visualize the results of the vectorial sum operation Figure 6.2.
The dot, or scalar, product of two vectors ($\avec{u}$,$\avec{v} \in \mathbb{R}^3$) is a scalar ($a \in \mathbb{R}$) equal to the sum of the products of the components of the vectors. Equivalently, it can be expressed as the product of the magnitudes of the two vectors times the cosine of the angle between them, $\phi \in [0,2\pi)$.
Scalar product \begin{align} \label{eq:vector-dot-product}\tag{12} \avec{u} \cdot \avec{v} = u_1v_1+u_2v_2+u_3v_3 = \|u\|\|v\|\cos(\phi) \in \mathbb{R} \end{align}
The dot product is a measure of the projection of vectors on one another (Figure 6.3).
When the two vectors are perpendicular, or orthogonal, the dot product is zero ($\cos(\pi/2) = 0$). This fact is often used as a test for orthogonality. Orthogonality is an important concept for linear spaces, as the most βefficientβ basis are orthogonal.
While the dot product depends on the metric chosen in the space (the Euclidian norm, in our case), the cross product even requires the definition of an orientation, or handedness.
Standard Basis In the Euclidian space $\mathbb{R}^3$, $\hat{\avec{i}}, \hat{\avec{j}}, \hat{\avec{k}}$ are the unit vectors for the standard basis, which is right handed.
In a right handed reference system such as the standard basis, the right hand rule (Figure 6.4) is the handy-est way to identify the direction of the vector resulting from a cross product.
ProTip: There is a valid reason for which it is called the right hand rule. Donβt use your left hand because you are holding a pen with the right one.
The cross, or vector, product between two vectors ($\avec{u}$, $\avec{v} \in \mathbb{R}^3$) is a vector that is orthogonal to each of the two vectors, hence is normal, or perpendicular, to the plane containing them. Its magnitude is given by the product of their magnitude times the sine of the angle between them, and its direction is indicated by the normal unit vector ($\hat{\avec{n}} \in \mathbb{R}^3$), identified by the right hand rule.
Vector product \begin{align} \label{eq:vector-cross-product}\tag{13} \avec{u} \times \avec{v} = [u_2v_3-u_3v_2, u_3v_1-u_1v_3, u_1v_2-u_2v_1]^T = \|u\|\|v\|\sin(\phi) \hat{\avec{n}}. \end{align}
Geometric interpretation A cross product encodes two pieces on information: a direction, which is orthogonal to the plane spanned by the two vectors, and a magnitude, which is equal to the area of the parallelogram having $\avec{u}$, and $\avec{v}$ as sides.
Keeping \eqref{eq:vector-cross-product} and Remark 7 - Geometric interpretation in mind, it should be intuitive to understand that: \begin{align} \label{eq:vec-cross-vv-v0}\tag{14} \avec{v} \times \avec{v} &= \avec{0}, \forall \avec{v} \in \mathbb{R}^n, \\ \avec{v} \times \avec{0} &= \avec{0}, \forall \avec{v} \in \mathbb{R}^n. \end{align}
The zero vector ($\avec{0}$) is a vector with zero magnitude, not the same as the number zero ($0$).
Each component of $\avec{w}$ is the difference of the products of the two other components of $\avec{u}$, and $\avec{v}$, in the order given by the chosen handedness of the basis. This combination resembles a cross (Figure 6.5), from which the name of cross product.
The components of a cross product can be computed through the Sarrus rule (see Section 7.6 - Determinant).
As consequence of the vectorial productβs definition and right handedness of the basis, the following hold true in the Cartesian space:
\begin{align} \label{eq:sb-cross-products}\tag{15} \hat{\avec{i}} \times \hat{\avec{j}} &= \hat{\avec{k}} \\ \hat{\avec{j}} \times \hat{\avec{k}} &= \hat{\avec{i}} \\ \hat{\avec{k}} \times \hat{\avec{i}} &= \hat{\avec{j}}. \end{align}
In this section we highlight the properties of vector operations, that derive from their definitions.
The vector sum obejs the following:
Letting $\phi \in [0,2\pi)$ be the angle between two vectors $\avec{u}, \avec{v}$, the dot product obejs the following:
Letting $\phi \in [0,2\pi)$ be the angle between two vectors $\avec{u}, \avec{v}$, the cross product obejs the following:
Linear dependance Two or more vectors $\{\avec{v_1},\cdots,\avec{v_n}\}$ are linearly dependant if there exists a set of scalars $\{a_1, \cdots, a_k\}, k \leq n$, that are not all zero, such that: $$ \label{eq:lin-dep}\tag{1} a_1\avec{v_1} + \cdots + a_k\avec{v_k} = \avec{0}. $$
When \eqref{eq:lin-dep} is true, it is possible to write at least one vector as a linear combination of the others.
Linear independance Two or more vectors ${\avec{v_1},\cdots,\avec{v_n}}$ are linearly independant if \eqref{eq:lin-dep} can be satisfied only by $k=n$ and $a_i =0, \forall i = 1, \cdots, n$.
Here we just add references to the suggested exercises, defined in the appropriate exercise chapters.
add exercises
In this section we have defined the fundamental concept of linearity and linear dependance. Moreover, we have introduced vectors, with their operations and algebraic properties.
Vectors and linearity are the base for understanding linear spaces, which are useful because they introduce some fundamental concepts related to the foundation of modeling of natural phenomena. Modeling will be invaluable in understanding the behavior of systems, and a powerful tool to predict future behaviors of the system, and control them when needed.
Jacopo
Jacopo
Dzenan
k:basic_math
k:linear_algebra
k:matrices
A matrix:
$$ \amat{A} = \left[ \begin{array}{ccc} a_{11} & \dots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{m1} & \dots & a_{mn} \end{array} \right] \in \reals^{m \times n} \label{eq:matrix}\tag{1} $$
is a table ordered by ($m$) horizontal rows and ($n$) vertical columns. Its elements are typically denoted with lower case latin letters, with subscripts indicating their row and column respectively. For example, $a_{ij}$ is the element of $A$ at the $i$-th row and $j$-th column.
A vector is a matrix with one column.
The number of rows and columns of a matrix are referred to as the matrix dimensions. $\amat{A} \in \reals^{m \times n}$ has dimensions $m$ and $n$.
Fat matrix When $n \gt{} m$, i.e., the matrix has more columns than rows, $\amat{A}$ is called fat matrix.
Tall matrix When $n \lt{} m$, i.e., the matrix has more rows than columns, $\amat{A}$ is called tall matrix.
Fat matrix When $n = m$, $\amat{A}$ is called square matrix.
Square matrices are particularly important.
Main diagonal
Secondary diagonal
Diagonal matrix A diagonal matrix has non zero elements only on its main diagonal. \begin{align} \amat{A} = \left[ \begin{array}{ccc} a_11 & 0 & \dots & 0 \\ 0 & a_22 & \ddots & \vdots \\ \vdots & \ddots & \ddots & 0 \\ 0 & \dots & 0 & a_nn \end{array} \right] \end{align}
Identity matrix An identity matrix is a diagonal square matrix with all elements equal to one. \begin{align} \amat{I} = \left[ \begin{array}{ccc} 1 & 0 & \dots & 0 \\ 0 & 1 & \ddots & \vdots \\ \vdots & \ddots & \ddots & 0 \\ 0 & \dots & 0 & 1 \end{array} \right] \end{align}
Null matrix The null, or Zero, matrix is a matrix whos elements are all zeros. \begin{align} \amat{0} = \left[ \begin{array}{ccc} 0 & \dots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \dots & 0 \end{array} \right] \end{align}
Dzenan
$\amat{A}\text{adj}(\amat{A}) = \text{det}(A)\amat{I}$
Square matrix:
$\amat{A}\amat{A}^{-1} = \amat{I}$
Exercise: Calculate a (square) Matrix Inverse
Exercise: Inverting a well-conditioned matrix (practice)
Exercise: Inverting an ill-conditioned matrix (practice)
Dzenan
for rectangular matrices (topic for advanced-linear-algebra?)
condition number of a matrix (?)
Dzenan
Dzenan
Dzenan
Dzenan
finish writing
Other metrics can be defined to measure the βlengthβ of a vector. Here, we report some commonly used norms. For a more in depth discussion of what constitutes a norm, and their properties, see:
[18].
Let $p \geq 1 \in \reals$. The $p$-norm is defined as:
$p$-norm \begin{align} \label{eq:vec-p-norm}\tag{1} \|\avec{v}\|_p = \displaystyle \left( \sum_{i=1}^{n} |v_i|^p \right)^{\frac{1}{p}}. \end{align}
The $p$-norm is a generalization of the $2$-norm ($p=2$ in \eqref{eq:p-norm}) introduced above (Definition 19 - Length of a vector). The following $1$-norm and $\infty$-norm can as well be obtained from \eqref{eq:vec-p-norm} with $p=1$ and $p \rightarrow \infty$ respectively.
The $1$-norm is the sum of the absolute values of a vectorβs components. It is sometimes referred to as the Taxicab norm, or Manhattan distance as it well describes the distance a cab has to travel to get from a zero starting point to a final destination $v_i$ on a grid.
$1$-norm Given a vector $\avec{v} \in \reals^n$, the $1$-norm is defined as: \begin{align} \label{eq:vec-one-norm}\tag{2} \|\avec{v}\| = \displaystyle \sum_{i=1}^{n}|v_i|. \end{align}
The infinity norm measures the maximum component, in absolute value, of a vector.
$\infty$-norm \begin{align} \label{eq:vec-inf-norm}\tag{3} \|\avec{v}\| = \displaystyle \max(|v_1|, \cdots, |v_n|). \end{align}
Miguel
Miguel
In this chapter we give a brief review of some basic probabilistic concepts. For a more in-depth treatment of the subject we refer the interested reader to a textbook such as [19].
The key underlying concept in probabilistic theory is that of an event, which is the output of a random trial. Examples of an event include the result of a coin flip turning up HEADS or the result of rolling a die turning up the number β4β.
Random Variable A (either discrete or continuous) variable that can take on any value that corresponds to the feasible output of a random trial.
For example, we could model the event of flipping a fair coin with the random variable $X$. We write the probability that $X$ takes HEADS as $p(X=\text{HEADS})$. The set of all possible values for the variable $X$ is its domain, $\aset{X}$. In this case, $$ \aset{X}=\{\text{HEADS},\text{TAILS}\}. $$ Since $X$ can only take one of two values, it is a binary random variable. In the case of a die roll, $$ \aset{X}=\{1,2,3,4,5,6\}, $$ and we refer to this as a discrete random variable. If the output is real value or a subset of the real numbers, e.g., $\aset{X} = \reals$, then we refer to $X$ as a continuous random variable.
Consider once again the coin tossing event. If the coin is fair, the have $p(X=HEADS)=p(X=TAILS)=0.5$. Here, the function $p(x)$ is called the probability mass function or pmf. The pmf is shown in Figure 16.1.
Here are some very important properties of $p(x)$: - $0\leq p(x) \leq (1)$ - $\sum_{x\in\aset{X}}=1$
In the case of a continuous random variable, we will call this function $f(x)$ and call it a probability density function, or pdf.
In the case of continuous RVs, technically the $p(X=x)$ for any value $x$ is zero since $\aset{X}$ is infinite. To deal with this, we also define another important function, the cumulative density function, which is given by $F(x) \triangleq p(X\leq x)$, and now we can define $f(x) \triangleq \frac{d}{dx}F(x)$. A pdf and corresponding cdf are shown in Figure 16.2 (This happens to be a Gaussian distribution, defined more precisely in Subsection 16.1.8 - The Gaussian Distribution.).
If we have two different RVs representing two different events $X$ and $Y$, then we represent the probability of two distinct events $x \in \aset{X}$ and $y \in \mathcal{Y}$ both happening, which we will denote as following: $p(X=x \; \text{AND} \; Y=y) = p(x,y)$. The function $p(x,y)$ is called joint distribution.
Again, considering that we have to RVs, $X$ and $Y$, imagine these two events are linked in some way. For example, $X$ is the numerical output of a die roll and $Y$ is the binary even-odd output of the same die roll. Clearly these two events are linked since they are both uniquely determined by the same underlying event (the rolling of the die). In this case, we say that the RVs are dependent on one another. In the event that we know one of events, this gives us some information about the other. We denote this using the following conditional distribution $p(X=x \; \text{GIVEN} \; Y=y) \triangleq p(x|y)$.
Write down the conditional pmf for the scenario just described assuming an oracle tells you that the die roll is even. In other words, what is p(x|\text{EVEN})?
(Warning: if you think this is very easy thatβs good, but donβt get over-confident.)
The joint and conditional distributions are related by the following (which could be considered a definition of the joint distribution):
\begin{equation} p(x,y) = p(x|y)p(y) \label{eq:joint}\tag{1} \end{equation}
and similarly, the following could be considered a definition of the conditional distribution:
\begin{equation} p(x|y) = \frac{p(x,y)}{p(y)} \; \text{if} \; p(y) \gt{} 0 \label{eq:condition}\tag{2} \end{equation}
In other words, the conditional and joint distributions are inextricably linked (you canβt really talk about one without the other).
If two variables are independent, then the following relation holds: $p(x,y)=p(x)p(y)$.
Upon closer inspection of \eqref{eq:joint}, we can see that the choice of which variable to condition upon is completely arbitrary. We can write:
$$ p(y|x)p(x) = p(x,y) = p(x|y)p(y) $$
and then after rearranging things we arrive at one of the most important formulas for mobile robotics, Bayesβ rule:
\begin{equation} p(x|y) = \frac{p(y|x)p(x)}{p(y)} \label{eq:bayes}\tag{3} \end{equation}
Exactly why this formula is so important will be covered in more detail in later sections (TODO), but we will give an initial intuition here.
Consider that the variable $X$ represents something that we are trying to estimate but cannot observe directly, and that the variable $Y$ represents a physical measurement that relates to $X$. We want to estimate the distribution over $X$ given the measurement $Y$, $p(x|y)$, which is called the posterior distribution. Bayesβ rule lets us to do this. For every possible state, you take the probability that this measurement could have been generated, $p(y|x)$, which is called the measurement likelihood, you multiply it by the probability of that state being the true state, $p(x)$, which is called the prior, and you normalize over the probability of obtaining that measurement from any state, $p(y)$, which is called the evidence.
From Wikipedia: Suppose a drug test has a 99% true positive rate and a 99% true negative rate, and that we know that exactly 0.5% of people are using the drug. Given that a personβs test gives a positive result, what is the probability that this person is actually a user of the drug.
Answer: $\approx$ 33.2%. This answer should surprise you. It highlights the power of the prior.
If we already have a joint distribution $p(x,y)$ and we wish to recover the single variable distribution $p(x)$, we must marginalize over the variable $Y$. The involves summing (for discrete RVs) or integrating (for continuous RVs) over all values of the variable we wish to marginalize:
\begin{align} p(x) &= \sum_{\mathcal{Y}} p(x,y) f(x) &= \int p(x,y) dy \end{align}
This can be thought of as projecting a higher dimensional distribution onto a lower dimensional subspace. For example, consider Figure 16.3, which shows some data plotted on a 2D scatter plot, and then the marginal histogram plots along each dimension of the data.
Marginalization is an important operation since it allows us to reduce the size of our state space in a principled way.
Two RVs, $X$ and $Y$ may be correlated, we may be able to encapsulate the dependence through a third random variable $Z$. Therefore, if we know $Z$
Is there a discussion of graphical models anywhere? Doing a good job of sufficiently describing graphical models and the dependency relations that they express requires careful thought. Without it, we should refer readers to a graphical models text (e.g., Koller and Friedman, even if it is dense)
The $n$th moment of an RV, $X$, is given by $E[X^n]$ where $E[]$ is the expection operator with:
$$ E[f(X)] = \sum_{\aset{X}} x \, f(x) $$ in the discrete case and $$ E[f(X)] = \int x\, f(x) dx $$ in the continuous case.
The 1st moment is the mean, $\mu_X=E[X]$.
The $n$th central moment of an RV, $X$ is given by $E[(X-\mu_X)^n]$. The second central moment is called the covariance, $\sigma^2_X=E[(X-\mu_X)^2]$.
The entropy of an RV is a scalar measure of the uncertainty about the value the RV.
A common measure of entropy is the Shannon entropy, whose value is given by
\begin{equation} H(X)=-E[\log_2 p(x)] \label{eq:shannon}\tag{4} \end{equation}
This measure originates from communication theory and literally represents how many bits are required to transmit a distribution through a communication channel. For many more details related to information theory we recommend [20].
As an example, we can easily write out the Shannon entropy associated with a binary RV (e.g. flipping a coin) as a function of the probability that the coin turns up heads (call this $p$):
\begin{equation} H(X) = -p\log_2 p - (1-p)\log_2 (1-p) \label{eq:binary_entropy}\tag{5} \end{equation}
Notice that our highest entropy (uncertainty) about the outcome of the coin flip is when it is a fair coin (equal probability of heads and tails). The entropy decays to 0 as we approach $p=0$ and $p=1$ since in these two cases we have no uncertainty about the outcome of the flip. It should also be clear why the function is symmetrical around the $p=0.5$ value.
In mobile robotics we use the Gaussian, or normal, distribution a lot.
The banana distribution is the official distribution in robotics! - AC
The banana distribution is Gaussian! http://www.roboticsproceedings.org/rss08/p34.pdf - LP
The 1-D Gaussian distribution pdf is given by:
\begin{equation} \mathcal{N}(x|\mu,\sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \label{eq:gaussian1D}\tag{6} \end{equation}
where $\mu$ is called the mean of the distribution, and $\sigma$ is called the standard deviation. A plot of the 1D Gaussian was previously shown in Figure 16.2.
We will rarely deal with the univariate case and much more often deal with the multi-variate Gaussian:
\begin{equation} \mathcal{N}(\state|\bmu,\bSigma) = \frac{1}{(2*\pi)^{D/2}|\bSigma|^{1/2}}\exp[-\frac{1}{2}(\state-\bmu)^T\bSigma^{-1}(\state - \bmu)] \label{eq:gaussianND}\tag{7} \end{equation}
The value from the exponent: $(\state-\bmu)^T\bSigma^{-1}(\state - \bmu)$ is sometimes written $||\state - \bmu||_\bSigma$ and is referred to as the Mahalanobis distance or energy norm.
Mathematically, the Gaussian distribution has some nice properties as we will see. But is this the only reason to use this as a distribution. In other words, is the assumption of Gaussianicity a good one?
There are two very good reasons to think that the Gaussian distribution is the βrightβ one to use in a given situation.
The central limit theorem says that, in the limit, if we sum an increasing number of independent random variables, the distribution approaches Gaussian
It can be proven (TODO:ref) that the Gaussian distribution has the maximum entropy subject to a given value for the first and second moments. In other words, for a given mean and variance, it makes the least assumptions about the other moments.
Exercise: derive the formula for Gaussian entropy
Harshit
Kinematics is the study of position, velocity and acceleration of geometric points.
A point has no dimensions. For example, the center of mass of a body is a point.
Harshit
In order to uniquely specify a position in some space, we use some numbers, coordinates, in a coordinate system. For example, in daily life, we would use the intersection of two streets, each with a unique name in the city, to specify the location of some cafe. If you find yourself in the ocean, you might communicate your location to someone by telling them the latitude and longitude readout on your GPS device. Generally speaking, a coordinate system provides us a way to denote any position in an unambiguous way. So you can communicate any position to another person, and by using the given coordinates, that person can arrive at the same exact position in space. Note, however, different coordinates can correspond to the same point.
Coordinate system A coordinate system is a surjective function mapping from a tuple of reals to some space $S$, respecting the local topology. (The local topology, which is beyond the scope of this chapter, roughly means that βnearbyβ points have coordinates βcloseβ together.)
Because the ability of naming or specifying a point in space is so fundamentally important, we often take that coordinates given by some coordinate system, often a Cartesian coordinate system, as the name of the point.
Consider the real number line $\reals$ as the space $S$. We can name an arbitrary point $x \in \reals$, which happens to be a real number, by itself. So to check with our definition, any two distinct points on the real line would have two distinct coordinates. Furthermore any point has a coordinate. Letβs call this coordinate system $A$.
One should note that the coordinate system given above is not the only possible way to name the points on a real line. We can assign coordinate $-x$ to the point $x \in \reals$ and obtain an equally valid coordinate system $B$. To be more specific, given a point $p$ with the coordinate $a$ in the first coordinate system $A$, we know that in the second coordinate system $B$, $p$ would have the coordinate $-a$. Therefore, there is a way to translate between coordinates in the two systems.
Consider the real plane $\reals^2$ as the space $S$. This is an important space within robotics and beyond. It can be used to represent images, with each pixel having its own position, or the location of your Duckiebot in Duckietown.
We can draw two perpendicular lines on the plane $S$. We call these two lines the $x$-axis and the $y$-axis. We assign $(0, 0)$ to the point of intersection, which we call the origin. Then we decide on the positive directions and units for each axis and the unit. We will use coordinates $(x, y)$ to specify a point located $x$-many units in the positive $x$-direction and $y$-many units in the positive $y$-direction away from the origin, respectively. If we draw the axis-parallel lines with integral $x$ coordinate and lines with integral $y$ coordinate, we obtain a visualization similar to that in Figure 19.1.
When representing the location of your Duckiebot in Duckietown, you might decide to choose a corner of the map as the origin and take east as the positive $x$-direction and north as the positive $y$-direction, and 1 meter as the unit length. In this case, a Duckiebot with location $(1, -2)$ would sit at 1 meter east and 2 meters south of the designated corner of the map.
Image space
It is customary to put the origin of an image at the top-left corner with the $x$-axis being horizontal and increasing to the right and the $y$-axis vertical and increasing downwards. In this way, the $x$ and $y$ coordinates index the column and row, respectively, of a particular pixel in the image. Such a convention is observed in OpenCV and other software libraries.
An alternative coordinate system for the plane is the polar coordinate system, where we specify a point by its direction and distance from a fixed reference point. To set up a polar coordinate system, you first decide on the pole, the reference point, then the polar axis, the reference direction. We will call the distance from the pole, the radial coordinate or radius, commonly denoted by $r$ or $\rho$, and the angle from the polar axis, the angular coordinate or polar angle, commonly denoted by $\phi$, $\varphi$ or $\theta$. See an example in Figure 19.2.
Note that in a polar coordinate system, a point has many equally valid names.
Exercise to readers: provide two such points and a few of their coordinates each.
Now, consider a Cartesian coordinate system $C$ whose origin is at the pole and its positive $x$-direction coincides with the polar axis. It is not hard to convert polar coordinates to Cartesian coordinates in $C$.
Exercise to readers: consult Figure 19.3 and write out the conversion formulae.)
Given the many options, you might wonder which coordinate system to use in any given situation. The answer is a practical one. Choose the one that helps simplify the problem at hand. As a trivial example, consider the equation for a unit circle. In a Cartesian coordinate system, it would be $x^2 + y^2 = 1$ whereas in a polar coordinate system, it would be much simpler: $r = 1$.
Exercise to readers: how about the equation for a straight line in polar coordinates?
This is an important space since we live in a three-dimensional world. Since many of our robots operate in this same world, many robots similarly represent coordinates in three dimensions, including unmanned aerial vehicles (UAVs) and autonomous underwater vehicles (AUVs).
Suppose the plane is the page or screen you are reading from, which is just a slice through the 3D space around it, we can extend a 2D Cartesian coordinate system on the plane to 3D by adding a $z$-axis that is perpendicular to the page, i.e., the $z$-, $x$-, and $y$-axis are mutually perpendicular. As done before, we need to choose a positive $z$-direction and there are two choices: coming out of the page or going into the page. They form the right-handed coordinate system or the left-handed coordinate system, respectively. We shall use right-handed coordinate systems unless otherwise noted. For more on handedness, see Wikipedia. Now the resulting coordinates become $(x, y, z)$, see Figure 19.4.
Similarly, we can extend the polar coordinate systems on the page or screen to 3D by defining a zenith direction (upwards) perpendicular to the plane, the polar angle to be the angle away from zenith, and the azimuth angle to be the orthogonal projection of a pointβs angle away from the polar axis on the plane. Together, a point has coordinates $(r, \theta, \phi)$ where $\theta$ denotes the polar angle and $\phi$, the azimuth angle. See Figure 19.5.
Exercise to readers: how to convert spherical coordinates to 3D Cartesian coordinates? and back?
If you find this topic interesting, there are many more coordinate systems than the ones covered here, such as the:
cylindrical coordinate system in 3D space and
parabolic coordinate system in 2D space.
Falcon Dai
Required Reading: The following assumes a working familiarity with 2D and 3D Cartesian coordinate systems. If you are not familiar with Cartesian coordinate systems, please read the chapter on coordinate systems.
Does Earth move around the sun or does the sun move around Earth? It turns out that this is an ill-posed question until we specify a frame of reference for measurement. For us, observers on Earth, it appears that the sun is moving. But for an observer on the moon, both Earth and the sun are moving. In general, motions are relative and we need a reference when measuring motion.
A reference frame, or just frame, is an abstract coordinate system and the set of physical reference points that uniquely specify the coordinate system (location of the origin and orientation). As a way of specifying the reference frame, we often say object $A$ is moving relative to object $B$, instead of specifying explicitly a reference frame $F_B$ that is attached to object $B$.
Suppose there are two cars A and B both moving at 60 miles per hour eastward. When saying this, we implicitly assume the reference frame of the ground. In the groundβs reference frame, the ground itself is at rest, car A and car B are both moving at 60 miles per hour eastward. In car Aβs reference frame, however, car B is at rest and the ground is moving 60 miles per hour westward!
To simplify the following discussion, we additionally assume that we choose reference frames such that their axes are parallel. In order to translate motions between reference frames, we assume two rules.
Exercise to readers: derive these rules from kinematics.
As a corollary to the first rule, we immediately derive that frame $R$ is at rest relative to itself, since $\avec{0}$ is the only vector is also its own opposite.
Exercise to readers: translate car Bβs motion relative to the ground to relative to car A.
Falcon Dai
Knowledge of k:tuple
Knowledge of time series.
Knowledge of time series operations, such as upsampling and downsampling.
A time series with domain $\aset{X}$ and time domain $\Time$ is a sequence of tuples $\tup{t_k, x_k} \in \Time \times \aset{X}$.
to write
to write
Super-dense time
Required Reading: The following assumes a working familiarity with 2D and 3D Cartesian reference frames. If you are not familiar with Cartesian reference frames, please read the chapter on reference frames. Some familiarity with linear alegra is also helpful.
Transformations are functions that map points to other points in space, i.e. $f: X \rightarrow X$. These maps are useful for describing motions over time. A particularly important class of transformations are linear transformations. These transformations can be represented by square matrices as they are linear and has the same domain and image.
Please refer to the Robotics Handbook section 1.2, and in the context of this course, readerβs goal is to attain a conceptual understanding, not necessarily knowing the exact formulae.
we need to provide links to the handbook. -AC
adapt the concepts in the context of Duckietown.
Falcon Dai
Introduction to type theory.
Knowledge of set theory.
Types are similar to sets in that they are collections that include some objects and exclude others. On a first glance, you can interpret $0 : \text{nat}$ as $0 \in \mathbb{N}$, reading the colon as membership in sets without any ill effect. But types and sets are the basic concepts belonging to two different formal systems, type theory and set theory, respectively. It is not essential to appreciate the difference in the scope of this course but for the curious readers, this section on Wikipedia can serve as a brief summary.
Given two types $A$ and $B$, we can construct the type $A \times B$, which we call their cartesian product. (Compare this with the similar concept of cartesian product of sets which is defined in terms of its elements and not a primitive.)
As we will be using ROS, which models a robotic system as a network of communicating programs. In order to understand each other, all the communicating programs talk to each other in well-defined message types. Message types in ROS are product types composed of primitive types and other message types.
Please read section 2 on ROS/msg page and answer: what are some primitive types in ROS? what are the fields and their types in message type Header?
Additionally, ROS/common_msgs page provides a list of pre-defined message types commonly used in robotics, such as Image (note how Header, a non-primitive type, is included in the definition) and Pose2D. As you have likely guessed, an RGB camera publishes Image messages, and a routing planning program might subscribe to the duckiebotβs current position in duckietown, as represented in a Pose2D message and calculates the appropriate wheel actions.
Historically, the flexibility of naive set theory allows for some paradoxical sets such as a set that contains all sets that does not contain itself. Does this set contains itself? This is known as Russellβs paradox which demonstrated that naive set theory is inconsistent. In response, Russell and colleagues developed type theory which demands all terms to be typed, i.e., to have a type, and used a hierarchy of types to avoid Russellβs paradox. Later, a subclass of type theories known as intuitionistic type theories internalized many key ideas in constructive mathematics and became a foundation for programming languages where computability is a major concern.
On a side note, this is not to say sets cannot serve as a formal foundation of mathematics. Russellβs paradox only shows that naive set theory is inconsistent. In fact, most working mathematicians today believe that the axiomatized Zermelo-Fraenkel set theory (together with the axiom of choice, usually abbreviated as ZFC) can serve as a βconsistentβ foundation of all mathematics.
Type theory is a fascinating subject in itself and recently, Homotopy Type Theory (HoTT) captured a lot of research interest. For more on the subject consult HoTT website. The first chapter of the HoTT book also provides a reasonable introduction to type theory. For more practical applications of these abstract ideas, you may be intrigued by the field of formal verification, where software is verified by mathematical proofs against the formal specification, automatically.
Falcon Dai
This unit will contain brief explanations / pointers to basic concepts of computer science.
to write
These are the theory units.
Liam
These days it is hard to separate the fact from the fiction when it comes to autonomous vehicles, particularly self-driving cars. Virtually every major car manufacturer has pledged to deploy some form of self-driving technology in the next five years. In addition, there are many startups and software companies which are also known to be developing self-driving car technology.
Hereβs a non-exhaustive list of some of companies that are actively developing autonomous cars:
Before even discussing any detailed notion of autonomy, we have to specify exactly what we are talking about. In the United States, the governing body is the NHTSA, and they have recently (Oct 2016) redefined the so-called βlevels of autonomyβ for self-driving vehicles.
In broad terms, they are as follows
Liam
This unit introduces some basic concepts ubiquitous in autonomous vehicle navigation.
The minimal basic backbone processing pipeline for autonomous vehicle navigation is shown in Figure 2.1.
For an autonomous vehicle to function, it must achieve some level of performance for all of these components. The level of performance required depends on the task and the required performance. In the remainder of this section, we will discuss some of the most basic options. In the next section we will briefly introduce some of the more advanced options that are used in state-of-the-art autonomous vehicles.
Sensor A sensor is a device that or mechanism that is capable of generating a measurement of some external phyiscal quantity
In general, sensors have two major types. Passive sensors generate measurements without affecting the environment that they are measuring. Examples include inertial sensors, odometers, GPS receivers, and cameras. Active sensors emit some form of energy into the environment in order to make a measurement. Examples of this type of sensor include Light Detection And Ranging (LiDAR), Radio Detection And Ranging (RaDAR), and Sound Navigation and Ranging (SoNAR). All of these sensors emit energy (from different spectra) into the environment and then detect some property of the energy that is reflected from the environment (e.g., the time of flight or the phase shift of the signal)
The raw data that is input from a sensor needs to be processed in order to become useful and even understandandable to a human.
First, calibration is usually required to convert units, for example from a voltage to a physical quantity. As a simple example consider a thermometer, which measures temperature via an expanding liquid (usually mercury). The calibration is the known mapping from amount of expansion of liquid to temperature. In this case it is a linear mapping and is used to put the markings on the thermometer that make it useful as a sensor.
We will distiguish between two fundamentally types of calibrations.
Intrinsic Calibration An intrinsic calibration is required to determine sensor-specific paramaters that are internal to a specific sensor.
Extrinsic Calibration An extrinsic calibration is required to determine the external configuration of the sensor with respect to some reference frame.
For more information about reference frames check out Unit G-20 - Reference frames
Calibration is very important consideration in robotics. In the field, the most advanced algorithms will fail if sensors are not properly calibrated.
Once we have properly calibrated data in some meaningful units, we often do some preprocessing to reduce the overall size of the data. This is true particularly for sensors that generate a lot of data, like cameras. Rather than deal with every pixel value generated by the camera, we will process an image to generate feature-points of interest. In βclassicalβ computer vision many different feature descriptors have been proposed (Harris, BRIEF, BRISK, SURF, SIFT, etc), and more recently Convolutional Neural Networks (CNNs) are being used to learn these features.
The important property of these features is that they should be as easily to associate as possible across frames. In order to achieve this, the feature descriptors should be invariant to nuissance parameters.
Now that we have used our sensors to generate a set of meaningful measurements, we need to combine these measurements together to produce an estimate of the underlying hidden state of the robot and possibly to environment.
State The state $\state_t \in \statesp$ is a sufficient statistic of the environment, i.e. it contains all sufficient information required for the robot to carry out its task in that environment. This can (and usually does) include the configuration of the robot itself.
What variables are maintained in the statespace $\statesp$ depends on the problem at hand. For example we may just be interested in a single robotβs configuration in the plane, in which case $\state_t \equiv \pose_t$. However, in other cases, such as simultaneous localization and mapping, me may also be tracking the map in the state space.
According to Bayesian principles, any system parameters that are not fully known and deterministic should be maintained in the state space.
In general, we do not have direct access to values in $\state$, instead we rely on our (noisy) sensor measurements to tell us something about them, and then we infer the values.
The animation in Figure 2.4 shows the lane following procedure. The output of the state estimator produces the green arrow in the top left pane.
The next inner loop of the nested controller in Figure 2.5 is the Vehicle Controller, which takes as input the reference trajectory generated by the Navigation and Motion Planning block and the current configuration of the robot, and uses the error between the two to generate a control signal.
The most basic feedback control law (See Unit H-27 - Feedback control is called PID (for proportional, integral, derivative) which will be discussed in Unit H-28 - PID Control. For an excellent introduction to this control policy see Figure 2.7.
We will also investigate some more advanced non-linear control policies such as Model Predictive Control, which is an optimization based technique.
The very innermost control loop deals with actually tracking the correct voltage to be sent to the motors. This is generally executed as close to the hardware level as possible. For example we have a Stepper Motor HAT See the parts list.
In general, we can make the autonomous navigation a simpler one by exploiting existing structure, infastructure, and contextual prior knowledge.
Infrastructure example: Maps or GPS satellites
Structure example: Known color and location of lane markings
Contextual prior knowledge example: Cars tend to follow the Rules of the Road
The basic building blocks enable static navigation in Duckietown. However, many other components are necessary for more realistic scenarios.
One key requirement is the ability to detect objects in the world such as but not limited to: signs, other robots, people, etc.
The simultaneous localization and mapping (SLAM) problem involves simultaneously estimating not only the robot state but also the map at the same time, and is a fundamental capability for mobile robotics. In autonomous driving, generally the most common application for SLAM is actual in the map-building task. Once a map is built then it can be pre-loaded and then used for pure localization. A demonstration of this in Duckietown is shown in Figure 2.9.
Other topics that will be covered include:
Visual-inertial navigation (VINS)
Fleet management and coordination
Scene segmentation
Deep perception
Text recognition
Andrea Censi
The many parts of a robotic system
Modern robotics development
Example of cloud learning, annotators in the loop scenario.
A robot is only a small part of a modern robotic system.
We briefly discuss what we could consider the possible components of a robotic system.
Of course, the robot, which includes:
The other robots with which the robot interacts. Perhaps they just need to avoid each other; perhaps they are collaborating.
The other machines. For example, a battery dock with that the robot can use to recharge
The infrastructure, including the network and off-board storage and computation resources.
The people, including:
While robotic systems have existed for decades, modern autonomous robotic systems are developed in a different way than traditional robotics/automation projects.
The classical model of development consists of:
These are instead the characteristics of modern robotics development model:
Variations on the following idea are what is typically implemented by autonomous vehicles developers for object detection.
The problem is to implement a machine-learning system that learns to perform an object detection task based on supervised learning.
Training and validation happen on large datasets that are continuously updated with new data coming from the cars, and new annotations coming from annotators.
We can consider βthe cloudβ as a component of a modern robotic system. The cloud can be modeled as a component that provides:
Because of the latency, it is not possible for real-time robotics applications to run completely on the cloud.
Therefore, the cloud is much more useful for tasks like the following:
As of 2017, the largest cloud-computing services are the ones offered by Amazon (AWS), Microsoft (Azure), Google (Google Cloud).
For supervised learning tasks, one needs to have large annotations databases.
The idea of using human annotators as a βsoftware serviceβ was first deployed on a large scale by Amazon with the βMechanical Turkβ project.
Nowadays, there exist companies that are specialized in providing annotations for AI tasks.
Examples of annotation services are:
Now that we have introduced the components, we will see how one can put everything together in a system (Figure 3.1).
This system works as follows:
In the near future, it might be that the design of robotic systems might become even more complicated. For example, it might be that blockchain technologies will allow machines to trade between them.
A self-driving car realizes it is too dirty; by itself, it finds a robotic carwash, and together they agree on the time and the price, and the car pays by itself using Bitcoin.
The robot is but a small part of a robotic system.
Development methods have changed recently: data is very important, as well as delocalized computation.
One of the cloud robotics papers XXX
Mechanical Turk XXX
A Blockchain tutorial XXX
Andrea Censi
Physical and logical architectures.
Deployment as mapping a physical architecture onto the logical.
Basic graph concepts.
Basic operating systems concepts.
When we design a robotic system, or any cyber-physical system, we distinguish between what we shall call βlogical architectureβ and the βphysical architectureβ.
The logical architecture describes how the functionality is divided in abstract modules, and what these modules communicate.
The physical architecture describes how the modules are instantiated. For example, this includes the information of how many processors are used, and which processor runs which routine.
The logical architecture is independent of the implementation (the hardware, the language.)
The logical architecture describes:
A logical architecture would also describe what are the representations used. This is explored more fully in the unit about representations.
Brief discussion of actor model
Processors
Buses / networks
Middleware
Orchestrators
In ROS that is the roscore program.
For deployment, one must choose how the logical architecture is mapped on the physical architecture.
This can be seen as a graph mapping problem.
One can define a computation graph as a graph where nodes are algorithms and edges are events.
A resource graph is a graph where nodes are processors and edges are communication channels.
Given a computation graph and a resource graph one must choose where to put each node in the computation graph in the resource graph.
More formally, the assignment is called a graph homomorphism.
You will see a concrete example of different ways to map software components on logical architectures in one of the first exercises.
Duckietown uses ROS. In ROS, the components are called nodes. In ROS, the granularity is at the level of hosts rather than processors. In regular vanilla Linux, the kernel decides which physical processor executes which process at any time.
In ROS, the assignment of nodes to processors happens using a launch file.
By modifying the launch file we can choose the layout of the computation.
Typically you will encounter three ways to deploy a graph:
Running everything on the robot. This is the regular βautonomousβ mode.
Running everything on the robot, but orchestrating from the laptop. In this
case, the roscore program runs on a laptop, and the other components on the robot.
Running heavy computation on the laptop. In this mode, the heavy computation processes run on the laptop, while the actuation and sensing drivers run on the robot.
The logical architecture describes the system decomposition, independent of the implementation.
The physical architecture describes how is the computation physically realized.
There are multiple ways to map a computation graph onto a resource graph. This is something that is immediately useful to understand for rapid development.
Andrea Censi
Fundamentally, we do not know how to build good robotic architectures.β¨
Not a smooth evolution from previous practices
industrial robot -> self-driving car: a big gapβ¨
Matt
Required Reading: The following assumes working knowledge of 2D and 3D Cartesian coordinate systems, reference frames, and coordinate transformations. If you are not familiar with these topics, please see the following preliminary chapters.
Robots are embodied autonomous agents that interact with a physical environment.
As you read through this book, you will find that shared representations of the agent (i.e., the robot) and the environment in which it operates are fundamental to a robotβs ability to sense, plan, and actβthe capabilities that are key to making a robot a robot.
The state $\state_t \in \statesp$ is a representation that consists of a compilation of all knowledge about the robot and its environment that is sufficient both to perform a particular task as well as to predict the future. Of course, βpredict the futureβ is vague, and we can not expect the state to include knowledge to predict everything about the future, but rather what is relevant in the context of the task.
The state $\state_t \in \statesp$ is a representation of the robot and the environment that is sufficient to predict the future in the context of the task being performed.
I understand that explicitly referencing the task as relevant to the notion of state is odd, but I want to convey that the representation that we choose for the state does depend upon the task.
Now, letβs be a bit more formal with regards to what we mean by a state being βsufficientβ to predict the future. We are specifically referring to a state that exhibits the Markov property, i.e., that it provides a complete summary of the past (again, in the context of the task). Mathematically, we say that a state is Markov if the future state is independent of the past given the present state (technically, this corresponds to first-order Markov):
\begin{equation} p(x_{t+1} \vert x_t, a_t, x_{t-1}, a_{t-1}, \ldots, x_0, a_0) = p(x_{t+1} \vert x_t, a_t) \end{equation}
A state exhibits the Markov property if and only if the above holds.
I usually say that βstateβ is all that is needed to predict the future. The agent only needs to keep track of a smaller thing than the state to act optimally. There isnβt a good name to use; but it should be distinct from βstateβ.
I think that this oversimplifies things. What is it about the future that is being predicted? Certainly, the states used by autonomous systems arenβt sufficient to predict everything about the future (e.g., a self-driving car canβt predict whether a street light is going to come on, but that probably doesnβt matter).
Knowledge about the robot and its environment is often extracted from the robotβs multimodal sensor streams, such as wheel encoders and cameras. Consequently, one might choose to formulate the state as the collection of all of the measurements that the robot acquires over time. Indeed, the use of low-level sensor measurements as the representation of the state has a long history in robotics and artificial intelligence and has received renewed attention of-late, notably in the context of deep learning approaches to visuomotor policy learning.
However, while measurement history is a sufficient representation of the robot and its operating environment, it serves as a challenging definition of state.
First, raw measurements are redundant, both within a single observation and across successive measurements. For example, one doesnβt need to reason over all pixels in an image, let alone all pixels within successive images to understand the location of a street sign.
Second, raw measurements contain a large amount of information that is not necessary for a given task. For example, the pixel intensities that capture the complex diffusion light due to clouds convey information that is not useful for self-driving cars. Requiring algorithms that deal with state to reason over these pixels would be unnecessarily burdensome.
Third, measurement history is very inefficient: its size grows linearly with time as the robot makes new observations, and it may be computationally intractable to access and process such a large amount of data.
This motivates the desire for a minimal representation that expresses knowledge that is both necessary and sufficient for the robot to perform a given task. More concretely, we will consider parameterized (symbolic) formulations of the state and will prefer representations that involve as few parameters as possible, subject to the state being Markov and the constraints imposed by the task.
Metric spaces (namely, Euclidean spaces) constitute the most commonly adopted state space in robotics, be it through the use of feature-based parameterizations of the state or gridmap representation. However, it is not uncommon to define the state of the robot and its environment in terms of a topological space or a hybrid metric-topological space.
Importantly, the exteroceptive and proprioceptive sensor measurements from which the robotβs perception algorithms infer the state are noisy, the models that describe the motion of the robot and environment are error-prone, and many aspects of the state are not directly observable (e.g., your Duckiebot may not be able to βseeβ some of the other Duckiebots on the road due to occlusions or the cameras limited field-of-view). As a result, robots must reason over probabilistic models of the state, commonly referred to as the belief, in order to effectively mitigate this uncertainty.
Discuss conventions
The state of the robot typically includes its pose $\pose_t$, which specifies the position and orientation of a coordinate frame affixed to the robot relative to a fixed global coordinate frame commonly referred to as the βworld frameβ. The pose then defines the rigid-body rigid-body transformation between the two reference frames.
For rigid-body robots that operate in a plane ($\reals^2$), the pose $\pose \in \SEtwo$ consists of the Cartesian coordinates $(x,y)$ that specify the robotβs position and the angle $\theta$ that defines the robotβs orientation (yaw). For robots in $\reals^3$, including some ground platforms, aerial vehicles, and underwater vehicles, the pose $\pose \in \SEthree$ consists of three Cartesian coordinates $(x, y, z)$ that encode the robotβs position, and three Euler angles $\phi, \theta, \psi$ that specify the robotβs orientation.
In addition to the robotβs pose, it is often useful to include its linear and angular velocities as elements of the state, resulting in an additional set of three or six parameters for robots that operate in two dimensions and three dimensions, respectively.
Discuss specific robot state representation for Duckietown.
The two most commonly used metric representations of the robotβs environment are occupancy grid models and feature-based representations.
Occupancy grid representations discretize the world into a set of grid cells. Associated with each cell are its Cartesian coordinates in a global reference frame, $(x,y)$ in $\reals^2$ and $(x,y,z)$ in $\reals^3$, and a binary label that indicates whether the cell is occupied, where a value of one denotes that the cell is occupied.
pictures for these two? -AC
Feature-based models constitute the second commonly used environment representation. Feature-based representations model the environment as a collection of landmarks, and parameterize each in terms of the landmarkβs position ($(x,y)$ or $(x,y,z)$ in $\reals^2$ and $\reals^3$, respectively) and possibly its orientation.
Discuss:
Discuss specific environment representation for Duckietown.
Matthew Walter
Andrea Censi
Understanding of time series. See: Unit G-21 - Time series.
Understanding of the basic concepts of event-based processing and why it is different from periodic processing.
Understanding of latency, frequency.
The response to these challenges claims the reformulation of answers to fundamental questions referred to discrete-time systems: βwhen to sample,β βwhen to update control action,β or βwhen to transmit information.β XXX
One of the generic answers to these questions is covered by adoption of the event-based paradigm that represents a model of calls for resources only if it is really necessary in system operation. In event-based systems, any activity is triggered as a response to events defined usually as a significant change of the state of relevance. Most βman-madeβ products, such as computer networks, complex software architectures, and production processes are natural, discrete event systems as all activity in these systems is driven by asynchronous occurrences of events. XXX
In standard signal processing, the data is assumed to arrive periodically with a certain fixed interval; in robotics, it is common to work with streams of data that arrive irregularly.
As per Definition 38, a time series is a sequence $\tup{t_k, x_k} \in \Time \times \aset{X}$, where $\Time$ is time and $\aset{X}$ is the domain in which the signals take values.
If the time series is periodic, it means that $$ t_k = t_0 + \Delta t_k. $$
Therefore, periodic processing is a special case of event-based processing.
How
The sensor tells the controller when there is interesting data to process.
The controller tells the sensor when to send data.
to write
to write
to write
to write
Some events might be lost, because the network loses the packet.
Latency and frequency are to be considered two completely independent quantities.
Hereβs
Should you structure your application with periodic processing, or event-based processing? Hereβs a few things to keep in mind.
To have truly periodic processing, you need to have an operating system that is βreal-timeβ.
A real-time operating system will be able to schedule processing of data at given intervals.
Periodic processing is easier to analyze from the theoretical point of view.
There are situations where sometimes you prefer to lose packets.
Assumes: k:udp, k:tcp
A reference on all things event-based (control and signal processing) [23]
A simple discussion of event-based control by K. J. Γ strΓΆm [24]
For a couple of different possible models for event-based processing:
Andrea
Knowledge of middleware, marshalling, service discovery.
Reusable code
Abstraction
Assumes: k:code-reusability, k:code-abstraction, k:code-encapsulation.
Also known as βserializationβ. In Python, this is called βpicklingβ.
Service discovery: do I know who to contact
.local addresses
Deciding when to do some processing
ROS
DDS
LCM
to write
to write
Andrea Censi
Why do we want to separate things into modules?
Now we need interfaces
We call βcontractsβ the guarantees that modules give to each other.
We organize the discussion as follows:
to write
to write
to write
to write
e.g. reference frames
Reliability, probability
Andrea
How to deal with configuration effectively.
We use Python examples but the general concepts are applicable to any programming languages. Some of the idioms here are good for dynamic object-oriented languages.
We will look at the typical story of how one deals with parameters.
summary
At the beginning you have:
def f(x):
return x * 10 + 1
Then it becomes:
def f(x):
# return x * 10 + 1
return x * 11 + 1
The next step is making sure that the parameters are extracted from the code
After this step, the code looks much better:
def f(x):
a = 10
b = 1
y = x * a + b
return y
Why is it better?
Note that it is not true that having fewer lines of code means the code is better! Legibility first.
Code is for humans to read, and only tangentially for machines to interpret. -???
Python manifesto
The next step is recognizing the parameters in the interface of the function.
In this case, we might write something like this:
def f(x, a=10, b=1):
y = x * a + b
return y
The next step is encapsulation of the object.
parameters and defaults
For example, consider an image resizing function:
def resize(y, w, h):
y = cv2.imresize(x, (w, h))
return y
Probably the better way is the following.
We create an ImageResizer class that is parametrized.
class ImageResizer():
def __init__(self, w, h):
self.w, self.h = w, h
def __call__(self, x):
y = cv2.imresize(x, (self.w, self.h))
return y
Now we can call this object as follows:
image_filter = ImageResizer(w=200,h=320)
x = ...
y = image_filter(x)
Notice that now we have abstracted the interface from the implementation: after the object is created, we can call it an βimage filterβ, with the implication that it is βsomething that given an image into anotherβ.
For example, we can now write a generic apply_filter function:
In Duckietownβ¦
Now, we still have the problem of how the user specified these parameters. We need some sort of glue that goes from the user interface, which might be the command line interface, a graphical interface, or configuration files.
It makes sense that this functionality is implemented consistently across the system.
The next step is understanding that a user never wants to deal with single parameters, but rather we need to give names to entire configurations.
For example, we wight want to say something like:
image_resizer_large:
w: 640
h: 480
image_resizer_small:
w: 320
h: 320
From the point of view of the user, it is easy to experiment.
From the point of view of the developer, it promotes an approach in which one writes more generic code.
If there
apply_filters:
filter_names:
- image_resizer_large
- increase_contrast
- crop_bottom
For example, suppose that there is a magic function
def instance(name):
""" Returns an instantiation of the named object """
...
Then, one could implement a filter combinations like in the following code. The
initialization parameter is the names of the filters, which are then
instantiated and stored. The __call__() function calls all the filters in the
that where defined
class ApplyFilters():
def __init__(self, filter_names):
self.filters = map(instance, filter_names)
def __call__(self, image):
for f in self.filters:
image = f(image)
return image
We now look at the Duckietown solution for the previous problems.
The code is in EasyAlgo.
Obtaining a mathematical model of the Duckiebot is important in order to (a) understand its behavior and (b) design a controller to obtain desired behaviors and performances, robustly.
The Duckiebot uses an actuator (DC motor) for each wheel. By applying different torques to the wheels a Duckiebot can turn, go straight (same torque to both wheels) or stay still (no torque to both wheels). This driving configuration is referred to as differential drive.
In this section we will derive the model of a differential drive wheeled robot. The Duckiebot model will receive voltages as input (to the DC motors) and produce a configuration, or pose, as output. The pose describes unequivocally the position and orientation of the Duckiebot with respect to some Newtonian βworldβ frame.
Different methods can be followed to obtain the Duckiebot model, namely the Lagrangian or Newton-Euler, we choose to describe the latter as it arguably provides a clearer physical insight. Showing the equivalence of these formulations is an interesting exercises that the interested reader can take as a challenge. A useful resource for modeling of a Duckiebot may be found here [27].
Requires:k:reference_frames (inertial, body), k:intro-transformations (Cartesian, polar)
Suggested: k:intro-ODEs-to-LTIsys
k:diff-drive-robot-model
relabel inertial frame -> local frame; $(\cdot)^I \rightarrow (\cdot)^L$
We first briefly recapitulate on the (reference frames)[#reference-frames] that we will use to model the Duckiebot, with the intent of introducing the notation used throughout this chapter. It is important to note that we restrict the current analysis to the plane, so all of the following in defined in $\reals^2$.
To describe the behavior of a Duckiebot three reference frames will be used:
A βworldβ frame: a right handed fixed reference system with origin in some arbitrary point $O$. We will indicate variables expressed in this frame with a superscript $W$, e.g., $X^W$, unless there is no risk of ambiguity, in which case no superscript will be used.
An βinertialβ frame: a fixed reference system parallel to the βworldβ frame, that spans the plane on which the Duckiebot moves. We will denote its axis as $\{X_I, Y_I\}$, and it will have origin in point $A=(x_A, y_A)$, i.e., the midpoint of the robotβs wheel axle. We will indicate variables expressed in this frame with a superscript $I$, e.g., $X^I$, unless there is no risk of ambiguity, in which case no superscript will be used.
A body (or βrobotβ) frame: a local reference frame fixed with respect to the robot, centered in $A$ as well. The $x$ axis points in the direction of the front of the robot, and the $y$ axis lies along the axis between the wheels, so to form a right handed reference system. We denote the robot body frame with $\{X_R, X_R\}$. The same superscript convention as above will be used. The wheels will have radius $R$.
The robot is assumed to be a rigid body, symmetric, and $X_r$ coincides with axis of symmetry. Moreover, the wheels are considered identical and at the same distance, $L$, from the axle midpoint $A$.
Moreover:
The center of mass $C^W = (x_c, y_c)$ of the robot is on the $x_r$ axis, at a distance $c$ from $A$, i.e., ($C^R = (c, 0)$);
$X^r$ forms an orientation angle $\theta$ with the local horizontal.
These notations are summarized in Figure 11.1.
We briefly recapitulate on a few transformations that we will use throughout this chapter.
Let $\avec{x^I} = [x^I, y^I]$ be a vector represented in the inertial frame and $\avec{X^I} = [x^I, y^I, 1]^T$ be its augmented version. It is possible to express such vector in the world frame through a translation:
\begin{align} \label{eq:mod-translation-i2w}\tag{11} \avec{X^W}= \amat{T}\avec{X^R}, \end{align}
where the translation matrix $\amat{T}$ is defined as:
\begin{align} \amat{T} = \left[ \begin{array}{ccc} 1 & 0 & x_A \\ 0 & 1 & y_A \\ 0 & 0 & 1 \end{array} \right] \left[\begin{array}{c} x^I \\ y^I \\ 1 \end{array} \right]. \end{align}
In the Euclidian space, each translation preserves distances (norms), i.e., is an isometry. So, for example, a velocity expressed in the inertial frame will have the same magnitude as that velocity expressed in the world frame.
Let $\avec{x^R} = [x^R, y^R]^T$ be a vector represented in the robot frame and $\avec{X^R} = [x^R, y^R, 1]^T$ its augmented version. It is possible to express such vector in the inertial frame through a rotation:
\begin{align} \label{eq:mod-rotation-r2i}\tag{12} \avec{X^I}= \amat{R}(\theta)\avec{X^R}, \end{align}
where $\amat{R}(\theta) \in SO(2)$ is an orthogonal rotation matrix:
\begin{align} \label{eq:mod-rot-mat}\tag{13} \amat{R}(\theta) = \left[ \begin{array}{ccc} \cos\theta & -\sin \theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{array} \right]. \end{align}
The orthogonality condition implies that $\amat{R}^T(\theta)\amat{R}(\theta) = \amat{R}(\theta)\amat{R}^T(\theta) = \amat{I}$, hence: $$ \label{eq:mod-orthogonality-cond}\tag{1} \amat{R}^T(\theta) = \amat{R}^{-1}(\theta), $$ which is quite nice.
A corollary of \eqref{eq:mod-translation-i2w} and \eqref{eq:mod-rotation-r2i} is that the translations and rotations can be combined in a single transformation, because $\avec{X^W} = \amat{T}\avec{X^I} = \amat{T} \amat{R}(\theta)\avec{X^R}$. The combined transformation matrix is given by:
\begin{align} \label{eq:mod-rototranslation-mat}\tag{14} \amat{T} \amat{R}(\theta) = \left[ \begin{array}{ccc} \cos\theta & -\sin \theta & x_A \\sin\theta & \cos\theta & y_A \\ 0 & 0 & 1 \end{array} \right]. \end{align}
While kinematics studies the properties of motions of geometric (i.e., massless) points, dynamical modeling takes into account the actual material distribution of the system. Once mass comes into play, motion is the result of the equilibrium of external forces and torques with inertial reactions. While different approaches can be used to derive these equations, namely the Lagrangian or Newtonian approaches (former based on energy considerations, latter on equilibrium of generalized forces), we choose to follow the Newtonian one here for it grants, arguably, a more explicit physical intuition of the problem. Obviously both methods lead to the same results when the same hypothesis are made.
For starters, recalling that $C^r = (c, 0)$ is the center of mass of the robot, we define the relevant notations:
| $(v_u, v_w)$ | Longitudinal and lateral velocities of $C$, robot frame |
| $(a_u, a_w)$ | Longitudinal and lateral accelerations of $C$, robot frame |
| $(F_{u_R}, F_{u_L})$ | Longitudinal forces exerted on the vehicle by the right and left wheels |
| $(F_{w_R}, F_{w_L})$ | Lateral forces exerted on the vehicle by the right and left wheels |
| $(\tau_R, \tau_L)$ | Torques acting on right and left wheel |
| $\theta$, $\omega = \dot \theta$ | Vehicle orientation and angular velocity |
| $M$ | Vehicle mass |
| $J$ | Vehicle yaw moment of inertia with respect to the center of mass $C$ |
Figure 11.2 summarizes these notations.
Before deriving the dynamic model of the robot, it is useful to recall some elements of polar coordinates kinematics.
Let $\avec{r}(t)$ identify a point in the space from the inertial frame at distance $r(t)$ from the $A$.
You might want to refresh Euler formula to convince yourself about the following.
\begin{align} \label{eq:mod-polar-kin-deriv}\tag{15} \avec{r}(t) &= r(t) e^{j\theta(t)} \\ \avec{\dot r}(t) &= v_u(t) e^{j\theta(t)} + v_w(t) e^{j(\theta(t)+\frac{\pi}{2})} \\ \avec{\ddot r}(t) &= a_u(t) e^{j\theta(t)} + a_w(t) e^{j(\theta(t)+\frac{\pi}{2})}, \end{align}
with:
\begin{align} \label{eq:mod-polar-kin-coeff}\tag{16} v_u(t) &= \dot r(t)\\ v_w(t) &= r (t) \dot \theta (t)\\ a_u(t) &= \dot v_u - v_w \dot \theta (t) = \ddot r(t) - r \dot \theta^2(t)\\ a_w(t) &= \dot v_w + v_u \dot \theta (t) = 2 \dot r (t) \dot \theta (t) + r(t) \ddot \theta (t). \end{align}
Keeping \eqref{eq:mod-polar-kin-deriv} and \eqref{eq:mod-polar-kin-coeff} in mind, it is useful to note (for later use) that, letting $\avec{r}(t)$ identify the position of the center of mass $C$ in the inertial frame:
\begin{align} \label{eq:mod-C-world-pos}\tag{17} \left\{ \begin{array}{ll} x_C^W(t) &= x_A(t) + r(t) \cos\theta(t) \\ y_C^W(t) &= y_A(t) + r(t) \sin\theta(t) \end{array} \right., \end{align}
and therefore:
\begin{align} \label{eq:mod-A-dot-polar}\tag{18} \left\{ \begin{array}{ll} \dot x^I_A(t) &= x^W_C(t) - v_u(t) \cos\theta(t) + v_w(t) \sin\theta(t)\\ \dot y^I_A(t) &= y^W_C(t) - v_u(t) \sin\theta(t) - v_w(t) \cos\theta(t) \end{array} \right.. \end{align}
The next step, and definitely the most critical, is writing the free body diagram of the problem (Figure 11.2). In this analysis the only forces acting on the robot are those applied from the wheels to the vehicleβs chassis. It is important to note that the third passive wheel (omnidirectional or caster) is not being taken into account.
We derive the dynamic model by imposing the simultaneous equilibrium of forces along the longitudinal and lateral directions in the robot frame with the respective inertial forces, and of the moments around the vertical axis (coming out of the.. screen) passing through the center of mass of the robotic vehicle.
\begin{align} \label{eq:mod-dyn-equilibria}\tag{19} Ma_u(t) &= F_{u_L}+F_{u_R} \\ Ma_w(t) &= F_{w_L}+F_{w_R} \\ \ddot{\theta}(t) &= \frac{L}{J}(F_{u_R}-F_{u_L}) + \frac{c}{J}(F_{w_R}+F_{w_L}). \end{align}
By substituting the \eqref{eq:mod-polar-kin-coeff} in \eqref{eq:mod-dyn-equilibria}, the equilibrium equations are expressed in terms of accelerations of the center of mass in the robot frame:
\begin{align} \dot{v}_u(t) &= v_w(t) \dot{\theta}(t) + \frac{F_{u_L}+F_{u_R}}{M} \label{eq:mod-dyn-equilibria2a}\tag{20} \\ \dot{v}_w(t) &= -v_u(t) \dot{\theta}(t)+\frac{F_{w_L}+F_{w_R}}{M} \label{eq:mod-dyn-equilibria2b}\tag{21} \\ \ddot{\theta}(t) &= \frac{L}{J}(F_{u_R}-F_{u_L}) - \frac{c}{J}(F_{w_R}+F_{w_L}). \label{eq:mod-dyn-equilibria2c}\tag{22} \end{align}
This is a general dynamic model (in the sense of no kinematic constraints) of a differential drive robot under the geometric assumptions listed above. It is noted that it is a coupled and nonlinear model, not exactly a best case scenario (we like linear things, because there are plenty of tools to handle them). When using the general dynamic model above, it makes sense to associate the general kinematic model as well, given by:
\begin{align} \label{eq:mod-gen-kin-mod}\tag{23} \dot{x}_A(t) &= v_u(t) \cos\theta(t) - (v_w(t)-c \dot \theta)\sin\theta(t) \\ \dot{y}_A(t) &= v_u(t) \sin\theta(t) + (v_w(t)-c \dot \theta)\sin\theta(t). \end{align}
the above \eqref{eq:mod-gen-kin-mod} can be obtained by recalling on one side that translations are isometric transformations, and on the other side that:
\begin{align} \label{eq:mod-xCW}\tag{24} \left\{ \begin{array}{ll} x_A(t) &= x^W_C(t) - c \cos\theta(t) \\ y_A(t) &= y^W_C(t) - c \sin\theta(t) \end{array} \right. \end{align}
\begin{align} \label{eq:mod-xCI}\tag{25} \left\{ \begin{array}{ll} x_C^I(t) &= v_u(t) \cos\theta(t) - v_w(t) \sin\theta(t) \\ y_C^I(t) &= v_u(t) \sin\theta(t) + v_w(t) \cos\theta(t) \end{array} \right. \end{align}
Equation \eqref{eq:mod-gen-kin-mod} can be rewritten as: $$ \label{eq:mod-gen-kin-mod-better}\tag{2} \avec{v_A^I} = \amat{R}(\theta) \avec{v_A^R} $$, where: $$ \label{eq:mod-v_A^R}\tag{3} \avec{v_A^R} = [v_u(t), v_w(t) - c\dot\theta(t)]^T. $$
In order to simplify the model, we proceed to impose some kinematic constraints.
In this section we derive the kinematic model of a differential drive mobile platform under the assumptions of (a) no lateral slipping and (b) pure rolling of the wheels. We refer to these two assumptions as kinematic constraints.
The kinematic constraints are derived from two assumptions:
By inverting \eqref{eq:mod-rotation-r2i}, this constraint can be expressed through the inertial frame variables, yielding:
$$ \label{eq:mod-no-lat-slip-constraint-i}\tag{5} \dot y_A(t) \cos \theta(t) -\dot x_A(t) \sin \theta(t) = 0. $$
Imposing \eqref{eq:mod-no-lat-slip-constraint-i} on \eqref{eq:mod-A-dot-polar} results in:
$$ \label{eq:mod-no-lat-slip-constraint-v_w}\tag{6} v_w(t) = \dot y_C^I(t) \cos\theta(t) - \dot x_C^I(t) \sin\theta(t), $$
and by recalling that $C^R = (c,0)$:
$$ \label{eq:mod-no-lat-slip-constraint-C}\tag{7} \dot y_C^I(t) \cos\theta(t) - \dot x_C^I(t) \sin\theta(t) = c\dot\theta(t). $$
Hence, we obtain the strongest expression of this constraint:
$$ \label{eq:mod-no-lat-slip-final}\tag{8} v_w(t) = c\dot\theta(t), $$
and therefore:
$$ \label{eq:mod-no-lat-slip-final-dot}\tag{9} \dot v_w(t) = c\ddot\theta(t). $$
a simpler way of deriving \eqref{eq:mod-no-lat-slip-final-dot} is noticing, from \eqref{eq:mod-v_A^R}, that $\dot y_A^R = v_w(t) - c\dot\theta(t)$.
\begin{align} \label{eq:mod-pure-rolling}\tag{26} \left\{ \begin{array}{ll} v_{P,r} &= R \dot \varphi_{r} \\ v_{P,l} &= R \dot \varphi_{l} \end{array} \right.. \end{align}
Another notable consequence of this assumption is that, always in the robot frame, the full power of the motor can be translated into a propelling force for the vehicle in the longitudinal direction. Or, more simply, it allows to write:
$$ \label{eq:mod-force-and-torque}\tag{10} F_{u, (\cdot)}(t)R = \tau_{(\cdot)}(t), $$
where $\tau_{(\cdot)}(t)$ is the torque exerted by each motor on its wheel $(\cdot) = {l,r}$.
In a differential drive robot, controlling the wheels at different speeds generates a rolling motion of rate $\omega = \dot \theta$. In a rotating field there always is a fixed point, the center of instantaneous curvature (ICC), and all points at distance $d$ from it will have a velocity given by $\omega d$, and direction orthogonal to that of the line connecting the ICC and the wheels (i.e., the axle). Therefore, by looking at Figure 11.1, we can write:
\begin{align} \label{eq:mod-kin-1}\tag{27} \left\{ \begin{array}{l} \dot \theta (d-L) &= v_l \\ \dot \theta (d+L) &= v_r \end{array} \right., \end{align}
from which:
\begin{align} \label{eq:mod-kin-2}\tag{28} \left\{ \begin{array}{l} d &= L \frac{v_r + v_l}{v_r - v_l} \\ \dot \theta &= \frac{v_r - v_l}{2L} \end{array} \right.. \end{align}
A few observations stem from \eqref{eq:mod-kin-2}:
Moreover, a differential drive robot cannot move in the direction of the ICC, it is a singularity.
By recalling the no lateral slipping motion \eqref{eq:mod-no-lat-slip-constraint-r} hypothesis and the pure rolling constraint \eqref{eq:mod-pure-rolling}, and noticing that the translational velocity of $A$ in the robot frame is $v_A = \dot \theta d = (v_r+v_l)/2$ we can write:
\begin{align} \label{eq:mod-kin-3}\tag{29} \left\{ \begin{array}{l} \dot x_A^R &= R (\dot \varphi_R +\dot \varphi_L)/2 \\ \dot y_A^R &= 0 \\ \dot \theta &= \omega = R(\dot \varphi_R - \dot \varphi_L)/(2L) \end{array} \right., \end{align}
which in more compact yields the simplified forward kinematics in the robot frame:
\begin{align} \label{eq:mod-forward-kinematics-robot-frame}\tag{30} \left[ \begin{array}{c} \dot x_A^R \\ \dot y_A^R \\ \dot \theta \end{array} \right] = \left[ \begin{array}{cc} \frac{R}{2} & \frac{R}{2} \\ 0 & 0 \\ \frac{R}{2L} & -\frac{R}{2L} \\ \end{array} \right] \left[ \begin{array}{c} \dot \varphi_R \\ \dot \varphi_L \end{array} \right]. \end{align}
Finally, by using \eqref{eq:mod-rot-mat}, we can recast \eqref{eq:mod-forward-kinematics-robot-frame} in the inertial frame.
The simplified forward kinematics model of a differential drive vehicle is given by: \begin{align} \label{eq:mod-forward-kinematics-inertial-frame}\tag{31} \displaystyle \avec{\dot q}^I = \amat{R}(\theta) \left[ \begin{array}{c} \dot x_A^r \\ \dot y_A^r \\ \dot \theta \end{array} \right] = \left[ \begin{array}{cc} \frac{R}{2} \cos \theta & \frac{R}{2} \cos \theta \\ \frac{R}{2} \sin \theta & \frac{R}{2} \sin \theta \\ \frac{R}{2L} & -\frac{R}{2L} \\ \end{array} \right] \left[ \begin{array}{c} \dot \varphi_R \\ \dot \varphi_L \end{array} \right] = \left[ \begin{array}{cc} \cos \theta & 0 \\ \sin \theta & 0 \\ 0 & 1 \\ \end{array} \right] \left[ \begin{array}{c} v_A \\ \omega \end{array} \right]. \end{align}
By implementing the kinematic constraints formulations derived above, i.e., the no lateral slipping (\eqref{eq:mod-no-lat-slip-final-dot}) and pure rolling (\eqref{eq:mod-force-and-torque}) in the general dynamic model, it is straightforward to obtain:
\begin{align} \label{eq:mod-dyn-model-a}\tag{32} \dot{v}_u (t) &= c \dot \theta^2(t) + \frac{1}{RM} (\tau_R(t)+\tau_L(t)) \\ \dot v_w(t) &= c \dot\theta(t) \\ \ddot \theta &= - \frac{Mc}{Mc^2+J} \dot\theta(t)v_u(t) + \frac{L}{R(Mc^2+J)}(\tau_R(t)-\tau_L(t)) \end{align}
The equations governing the behavior of a DC motor are driven by an input armature voltage $V(t)$:
\begin{align} V(t) &= Ri(t) + L \frac{di}{dt} + e(t) \\ e(t) &= K_b \dot\varphi(t) \\ \tau_m(t) &= K_t i(t) \\ \tau(t) &= N \tau_m(t), \end{align}
where $(K_b, K_t)$ are the back emf and torque constants respectively and $N$ is the gear ratio ($N=1$ in the Duckiebot).
Figure 11.4 shows a diagram of a typical DC motor.
Having a relation between the applied voltage and torque, in addition to the dynamic and kinematic models of a differential drive robot, allows us to determine all possible state variables of interest.
torque disturbances acting on the wheels, such as the effects of friction, can be modeled as additive terms (of sign opposite to $\tau$) in the DC motor equations.
In this chapter we derived a model for a differential drive robot. Although several simplifying assumption were made, e.g., rigid body motion, symmetry, pure rolling and no lateral slipping - still the model is nonlinear.
Regardless, we now have a sequence of descriptive tools that receive as input the voltage signal sent by the controller, and produce as output any of the state variables, e.g., the position, velocity and orientation of the robot with respect to a fixed inertial frame.
Several outstanding questions remain. For example, we need to determine what is the best representation for our robotic platform - polar coordinates, Cartesian with respect to an arbitrary reference point? Or maybe there is a better choice?
Finally, the above model assumes the knowledge of a number of constants that are characteristic of the robotβs geometry, materials, and the DC motors. Without the knowledge of those constant the model could be completely off. Determination of these parameters in a measurement driven way, i.e., the βsystem identificationβ of the robotβs plant, is subject of the odometry class.
Jacopo
Matt
For the interested reader, there are several excellent books that discuss fundamental topics in computer vision. These include (in no particular order), Computer Vision: Algorithims and Applications (available online), Computer Vision: A Modern Approach and Multiple View Geometry in Computer Vision.
Topics:
Matt
We could break this up into separate sub-sections for (prospective) projection and lenses.
Topics:
Matt
The discussion of intrinsic and extrinsic models could be moved up to the geometry subsection
Topics
Matt
Background: Discrete-time signal processing
We currently donβt have a preliminary section on signal processing and itβs probably sufficient to point readers to a good reference (e.g., Oppenheim and Schafer).
Topics
May want to stick to linear filtering
Matt
Matt
Topics:
Matt
Matt
Liam
Markov Property
Bayes Filter
Graphical representation
Kalman Filter
Extended Kalman Filter
Liam
MAP estimation
Laplace Approximation
Cholesky Decomposition?
Incrementalization - Givens Rotations - iSAM
ETH
ETH
ETH
ETH
Jacopo
Jacopo
Jacopo
Nick and David
Nick and David
Nick and David
Jacopo
Nick and David
Nick
Liam
Liam
Liam
Liam
ETH
ETH
pdf
supp. materialbibtex
Bruno Siciliano and Oussama Khatib. Springer Handbook of Robotics. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2007. Β Β
Duckietown: an innovative way to teach autonomy.
In EduRobotics 2016. Athens, Greece, December 2016.
pdf
supp. materialbibtex
Tosini, G., Ferguson, I., Tsubota, K. Effects of blue light on the circadian system and eye physiology. Molecular Vision, 22, 61β72, 2016 (online).
Albert Einstein. Zur Elektrodynamik bewegter KΓΆrper. (German) On the electrodynamics of moving bodies. Annalen der Physik, 322(10):891β921, 1905. Β Β DOIΒ Ivan Savov. Linear algebra explained in four pages. https://minireference.com/static/tutorials/linear_algebra_in_4_pages.pdf, 2017. Online; accessed 23 August 2017. Β Β KaareΒ Brandt Petersen and MichaelΒ Syskind Pedersen. The matrix cookbook. Β Β .pdfΒ Matrix (mathematics). Matrix (mathematics) β Wikipedia, the free encyclopedia, 2017. Online; accessed 2-September-2017. Β Β www:Β PeterΒ D. Lax. Functional Analysis. Wiley Interscience, New York, NY, USA, 2002. Β Β Athanasios Papoulis and S.Β Unnikrishna Pillai. Probability, Random Variables and Stochastic Processes. McGraw Hill, fourth edition, 2002. Β Β David J.Β C. MacKay. Information Theory, Inference & Learning Algorithms. Cambridge University Press, New York, NY, USA, 2002. Β Β H.Β Choset, W.Β Burgard, S.Β Hutchinson, G.Β Kantor, L.Β E. Kavraki, K.Β Lynch, and S.Β Thrun. Principles of robot motion: Theory, agorithms, and implementation. MIT Press, June 2005. Β Β StevenΒ M. LaValle. Planning Algorithms. Cambridge University Press, New York, NY, USA, 2006. Β Β M.Β Miskowicz. Event-Based Control and Signal Processing. Embedded Systems. CRC Press, 2015. Β Β httpΒ KarlΒ J. AstrΓΆm. Event Based Control, pages 127β147. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. Β Β DOIΒ httpΒ Summary. In spite of the success of traditional sampled-data theory in computer control it has some disadvantages particularly for distributed, asynchronous, and multi-rate system. Event based sampling is an alternative which is explored in this paper. A simple example illustrates the differences between periodic and event based sampling. The architecture of a general structure for event based control is presented. The key elements are an event detector, an observer, and a control signal generator, which can be regarded as a generalized data-hold. Relations to nonlinear systems are discussed. Design of an event based controller is illustrated for a simple model of a micro-mechanical accelerometer.
EdwardΒ A Lee and DavidΒ G Messerschmitt. Synchronous data flow. Proceedings of the IEEE, 75(9):1235β1245, 1987. Β Β Rajit Manohar and KΒ Mani Chandy. Delta-dataflow networks for event stream processing. pages 1β6, June 2010. Β Β RomanoΒ M DeSantis. Modeling and path-tracking control of a mobile wheeled robot with a differential drive. Robotica, 13(4):401β410, 1995. Β Β Richard Szeliski. Computer Vision: Algorithms an Applications. Springer, 2010. Β Β httpΒ David Forsyth and Jean Ponce. Computer Vision: A Modern Approach. Prentce Hall, Upper Saddle River, NJ, 2011. Β Β Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, second edition, 2004. Β Β
These exercises can guide you from the status of a novice coder to experienced roboticist.
Andrea Daniele
duckietown_utils APIs for reading/writing images.Create an implementation of the program dt-image-flip0, specified below.
If this exercise is too easy for you, skip to Unit I-2 - Exercise: Basic image operations, adult version.
dt-image-flip0The program dt-image-flip0 takes as an argument a JPG file with extension .jpg:
$ dt-image-flip0 file.jpg
and creates a file called file.flipped.jpg that is flipped around the horizontal axis.


dt-image-flip.
The OpenCV library provides a utility function called imread
that loads an image from a file.
This is the kind of thing that they need to figure out how to do with pixels. -AC
The duckietown_utils
package provides the utility function write_image_as_jpg()
that writes an image to a JPEG file.
Andrea Daniele
Implement the program dt-image-flip specified in the following section.
This time, we specify exactly what should happen for various anomalous conditions. This allows to do automated testing of the program.
dt-image-flip specificationThe program image-ops expects exactly two arguments: a filename (a JPG file)
and a directory name.
$ dt-image-flip file outdir
If the file does not exist, the script must exit with error code 2.
If the file cannot be decoded, the script must exit with error code 3.
If the file exists, then the script must create:
outdir/regular.jpg: a copy of the initial fileoutdir/flip.jpg: the file, flipped vertically.outdir/side-by-side.jpg: the two files, side by side.If any other error occurs, the script should exit with error code 99.


flip.jpg

side-by-side.jpg
image-ops.
Good explanation, but shouldnβt it go in the previous exercise? -AC
An image loaded using the OpenCV function
imread
is stored in memory as a
NumPy array.
For example, the image shown above (Figure 2.1a -
The original picture.
) will be represented in
memory as a NumPy array with shape (96, 96, 3). The first dimension indicates
the number of pixels along the Y-axis, the second indicates the number of pixels
along the X-axis and the third is known as number of channels (e.g., Blue,
Green, and Red).
NumPy provides a utility function called
concatenate
that joins a sequence of arrays along a given axis.
image-ops-testerWe provide 4 scripts that can be used to make sure that you wrote a conforming dt-image-flip.
The scripts are image-ops-tester-good, image-ops-tester-bad1, image-ops-tester-bad2, and image-ops-tester-bad3.
You can find them in the directory
/exercises/dt-image-flip/image-ops-tester
in the duckietown/duckuments repository.
The script called image-ops-tester-good tests your program in a situation in which we expect it to work properly.
The 3 βbadβ test scripts (i.e., image-ops-tester-bad1 through image-ops-tester-bad3) test your code in
situations in which we expect your program to complain in the proper way.
Use them as follows:
$ image-ops-tester-scenario candidate-program
The tester scripts must be called from their own location. Make sure to change your working directory to
/exercises/dt-image-flip/image-ops-tester before launching the tester scripts.
If the script cannot be found, image-ops-tester-scenario will return 1.
image-ops-tester-scenario will return 0 if the program exists and conforms
to the specification (Section 2.3 - dt-image-flip specification).
If it can establish that the program is not good, it will return 11.
Things that are not tested are broken.
Andrea Daniele
Create an implementation of dt-bag-analyze according to the specification below.
dt-bag-analyzeCreate a program that summarizes the statistics of data in a bag file.
$ dt-bag-analyze bag file
Compute, for each topic in the bag:
Print out the statistics using the YAML format. Example output:
$ dt-bag-analyze bag file
"topic name":
num_messages: value
period:
min: value
max: value
average: value
median: value
A bag is a file format in ROS for storing ROS message data. The package rosbag
defines the class Bag
that provides all the methods needed to serialize messages to and from a single
file on disk using the bag format.
In ROS the time is stored as an object of type
rostime.Time.
An object t, instance of
rostime.Time,
represents a time instant as the number of
seconds since epoch (stored in t.secs) and the number of nanoseconds since
t.secs (stored in t.nsecs). The utility function
t.to_sec()
returns the time (in seconds) as a floating number.
Download the ROS bag
example_rosbag_H3.bag.
Run your program on it and compare the results:
$ dt-bag-analyze example_rosbag.bag
/tesla/camera_node/camera_info:
num_messages: 653
period:
min: 0.01
max: 0.05
average: 0.03
median: 0.03
/tesla/line_detector_node/segment_list:
num_messages: 198
period:
min: 0.08
max: 0.17
average: 0.11
median: 0.1
/tesla/wheels_driver_node/wheels_cmd:
num_messages: 74
period:
min: 0.02
max: 4.16
average: 0.26
median: 0.11
Andrea Daniele
Implement the program dt-bag-decimate as specified below.
dt-bag-decimateThe program dt-bag-decimate takes as argument a bag filename, an integer
value greater than zero, and an output bag file:
$ dt-bag-decimate "input bag" n "output bag"
The output bag contains the same topics as the input bag, however, only 1 in
n messages from each topic are written. (If n is 1, the output is the same as the input.)
In ROS, a new bag can be created by specifying the mode w (i.e., write) while
instantiating the class rosbag.Bag.
For example:
from rosbag import Bag
new_bag = Bag('./output_bag.bag', mode='w')
Visit the documentation page for the class rosbag.Bag for further information.
A ROS bag instantiated in write mode accepts messages through the function
write().
To check that the program works, you can compute the statistics
of the data using the program dt-bag-analyze that you have created
in Unit I-3 - Exercise: Simple data analysis from a bag.
You should see that the statistics have changed.
Andrea Daniele
Write a program dt-bag-thumbnails as specified below.
dt-bag-thumbnailsThe program dt-bag-thumbnails creates thumbnails for some image stream
topic in a bag file.
The syntax is:
$ dt-bag-thumbnails bag topic output_dir
This should create the files:
output_dir/00000.jpg
output_dir/00001.jpg
output_dir/00002.jpg
output_dir/00003.jpg
output_dir/00004.jpg
...
where the progressive number is an incremental counter for the frames.
If you donβt have a ROS bag to work on, you can download the test bag
example_rosbag_H5.bag.
You should be able to get a total of 653 frames out of it.
The duckietown_utils
package provides the utility function rgb_from_ros()
that processes a ROS message and returns the RGB image it contains (if any).
In OpenCV, an image can be converted from one color space (e.g., BGR) to another
supported color space (e.g., RGB). OpenCV provides a list of supported
conversions. A ColorConversionCode defines a conversion between two different
color spaces. An exhaustive list of color conversion codes can be found
here.
The conversion from a color space to another is done with the function
cv.cvtColor.
Andrea Daniele
Create dt-instagram as specified below.
dt-instagramWrite a program dt-instagram that applies a list of filters to an image.
The syntax to invoke the program is:
$ dt-instagram image in filters image out
where:
image in is the input image;filters is a string, which is a colon-separated list of filters;image out is the output image.The list of filters is given in Subsection 6.3.1 - List of filters.
For example, the result of the command:
$ dt-instagram image.jpg flip-horizontal:grayscale out.jpg
is that out.jpg contains the input image, flipped and than converted to grayscale.
Because these two commute, this command gives the same output:
$ dt-instagram image.jpg grayscale:flip-horizontal out.jpg
Here is the list of possible values for the filters, and their effect:
flip-vertical: flips the image verticallyflip-horizontal: flips the image horizontallygrayscale: Makes the image grayscalesepia: make the image sepiaIn OpenCV it is possible to define custom filters and apply them to an image.
A linear filter (e.g., sepia) is defined by a linear 9-dimensional kernel.
The sepia filter is defined as:
$$ K_{sepia} = \begin{bmatrix} 0.272 & 0.534 & 0.131 \newline 0.349 & 0.686 & 0.168 \newline 0.393 & 0.769 & 0.189 \end{bmatrix} $$
A linear kernel describing a color filter defines a linear transformation in the
color space. A transformation can be applied to an image in OpenCV by using the function
transform().
Andrea Daniele
Create dt-bag-instagram as specified below.
dt-bag-instagramWrite a program dt-bag-instagram that applies a filter to a stream of images
stored in a ROS bag.
The syntax to invoke the program is:
$ dt-bag-instagram bag in topic filters bag out
where:
- `bag in` is the input bag;
- `topic` is a string containing the topic to process;
- `filters` is a string, which is a colon-separated list of filters;
- `bag out` is the output bag.
If you donβt have a ROS bag to work on, you can download the test bag
example_rosbag_H5.bag.
sensor_msgs/CompressedImage messageThe duckietown_utils
package provides the utility function
d8_compressed_image_from_cv_image()
that takes a BGR image, compresses it and wraps it into a sensor_msgs/CompressedImage
ROS message.
Play your bag out ROS bag file and run the following command to make sure
that your program is working.
$ rosrun image_view image_view image:=topic _image_transport:=compressed
Andrea Daniele
You may find useful: Unit K-6 - Minimal ROS node - pkg_name.
That tutorial is about listening to text messages and writing back
text messages. Here, we apply the same principle, but to images.
Create a ROS node that takes camera images and applies a given operation, as specified in the next section, and then publishes it.
dt_live_instagram_robot name_nodeCreate a ROS node dt_live_instagram_robot name_node that takes a parameter called filter, where the filter is something from the list Subsection 6.3.1 - List of filters.
You should launch your camera and joystick from β~/duckietownβ with
duckiebot $ make demo-joystick-camera
Then launch your node with
duckiebot $ roslaunch dt_live_instagram_robot name dt_live_instagram_robot name_node.launch filter:=filter
This program should do the following:
Subscribe to the camera images, by finding
a topic that is called .../compressed. Call the name of the
topic topic (i.e., topic = ...).
Publish to the topic topic/filter/compressed a stream of images (i.e., video frames)
where the filter is applied to the images.
$ rqt_image_view
and look at topic/filter/compressed
Jonathan Michaux and Dzenan Lapandic
During the lectures, we have explained one direction of the image pipeline:
image -> [feature extraction] -> 2D features -> [ground projection] -> 3D world coordinates
In this exercise, we are going to look at the pipeline in the opposite direction.
It is often said that:
βThe inverse of computer vision is computer graphics.β
The inverse pipeline looks like this:
3D world coordinates -> [image projection] -> 2D features -> [rendering] -> image
dt_augmented_reality.Then verify the results in the following 3 situations.
dt_augmented_reality with map file calibration_pattern.yaml.(Adjust the position until you get perfect match of reality and augmented reality.)
dt_augmented_reality with map file lane.yaml.(Adjust the position until you get a perfect match of reality and augmented reality.)
dt_augmented_reality with map file intersection_4way.yaml.(Adjust the position until you get a perfect match of reality and augmented reality.)
Submit the images according to location-specific instructions.
dt_augmented_realityIn this assignment you will be writing a ROS package to perform the augmented reality exercise. The program will be invoked with the following syntax:
$ roslaunch dt_augmented_reality-robot name augmenter.launch map_file:=map file robot_name:=robot name local:=1
where map file is a YAML file containing the map (specified in Section 9.5 - Specification of the map).
If robot name is not given, it defaults to the hostname.
The program does the following:
/robot name/camera_node/image/compressed.It reads each image, projects the map features onto the image, and then writes the resulting image to the topic
/![robot name]/AR/![map file basename]/image/compressed
where map file basename is the basename of the file without the extension.
We provide you with ROS package template that contains the AugmentedRealityNode. By default, launching the AugmentedRealityNode should publish raw images from the camera on the new /robot name/AR/map file basename/image/compressed topic.
In order to complete this exercise, you will have to fill in the missing details of the Augmenter class by doing the following:
process_image that undistorts raw images.ground2pixel that transforms points in ground coordinates (i.e. the robot reference frame) to pixels in the image.callback that writes the augmented image to the appropriate topic.The map file contains a 3D polygon, defined as a list of points and a list of segments that join those points.
The format is similar to any data structure for 3D computer graphics, with a few changes:
Here is an example of the file contents, hopefully self-explanatory.
The following map file describes 3 points, and two lines.
points:
# define three named points: center, left, right
center: [axle, [0, 0, 0]] # [reference frame, coordinates]
left: [axle, [0.5, 0.1, 0]]
right: [axle, [0.5, -0.1, 0]]
segments:
- points: [center, left]
color: [rgb, [1, 0, 0]]
- points: [center, right]
color: [rgb, [1, 0, 0]]
The reference frames are defined as follows:
axle: center of the axle; coordinates are 3D.camera: camera frame; coordinates are 3D.image01: a reference frame in which 0,0 is top left, and 1,1 is bottom right of the image; coordinates are 2D.(Other image frames will be introduced later, such as the world and tile reference frame, which
need the knowledge of the location of the robot.)
RGB colors are written as:
[rgb, [R, G, B]]
where the RGB values are between 0 and 1.
Moreover, we support the following strings:
red is equivalent to [rgb, [1,0,0]]green is equivalent to [rgb, [0,1,0]]blue is equivalent to [rgb, [0,0,1]]yellow is equivalent to [rgb, [1,1,0]]magenta is equivalent to [rgb, [1,0,1]]cyan is equivalent to [rgb, [0,1,1]]white is equivalent to [rgb, [1,1,1]black is equivalent to [rgb, [0,0,0]]hud.yamlThis pattern serves as a simple test that we can draw lines in image coordinates:
points:
TL: [image01, [0, 0]]
TR: [image01, [0, 1]]
BR: [image01, [1, 1]]
BL: [image01, [1, 0]]
segments:
- points: [TL, TR]
color: red
- points: [TR, BR]
color: green
- points: [BR, BL]
color: blue
- points: [BL, TL]
color: yellow
The expected result is to put a border around the image: red on the top, green on the right, blue on the bottom, yellow on the left.
calibration_pattern.yamlThis pattern is based off the checkerboard calibration target used in estimating the intrinsic and extrinsic camera parameters:
points:
TL: [axle, [0.315, 0.093, 0]]
TR: [axle, [0.315, -0.093, 0]]
BR: [axle, [0.191, -0.093, 0]]
BL: [axle, [0.191, 0.093, 0]]
segments:
- points: [TL, TR]
color: red
- points: [TR, BR]
color: green
- points: [BR, BL]
color: blue
- points: [BL, TL]
color: yellow
The expected result is to put a border around the inside corners of the checkerboard: red on the top, green on the right, blue on the bottom, yellow on the left.
lane.yamlWe want something like this:
0
| | | . | | |
| | | . | | |
| | | . | | |
| | | . | | |
| | | . | | |
| | | . | | |
WW L WY L WW
1 2 3 4 5 6
Then we have:
points:
p1: [axle, [0, 0.2794, 0]]
q1: [axle, [D, 0.2794, 0]]
p2: [axle, [0, 0.2286, 0]]
q2: [axle, [D, 0.2286, 0]]
p3: [axle, [0, 0.0127, 0]]
q3: [axle, [D, 0.0127, 0]]
p4: [axle, [0, -0.0127, 0]]
q4: [axle, [D, -0.0127, 0]]
p5: [axle, [0, -0.2286, 0]]
q5: [axle, [D, -0.2286, 0]]
p6: [axle, [0, -0.2794, 0]]
q6: [axle, [D, -0.2794, 0]]
segments:
- points: [p1, q1]
color: white
- points: [p2, q2]
color: white
- points: [p3, q3]
color: yellow
- points: [p4, q4]
color: yellow
- points: [p5, q5]
color: white
- points: [p6, q6]
color: white
intersection_4way.yamlpoints:
NL1: [axle, [0.247, 0.295, 0]]
NL2: [axle, [0.347, 0.301, 0]]
NL3: [axle, [0.218, 0.256, 0]]
NL4: [axle, [0.363, 0.251, 0]]
NL5: [axle, [0.400, 0.287, 0]]
NL6: [axle, [0.409, 0.513, 0]]
NL7: [axle, [0.360, 0.314, 0]]
NL8: [axle, [0.366, 0.456, 0]]
NC1: [axle, [0.372, 0.007, 0]]
NC2: [axle, [0.145, 0.008, 0]]
NC3: [axle, [0.374, -0.0216, 0]]
NC4: [axle, [0.146, -0.0180, 0]]
NR1: [axle, [0.209, -0.234, 0]]
NR2: [axle, [0.349, -0.237, 0]]
NR3: [axle, [0.242, -0.276, 0]]
NR4: [axle, [0.319, -0.274, 0]]
NR5: [axle, [0.402, -0.283, 0]]
NR6: [axle, [0.401, -0.479, 0]]
NR7: [axle, [0.352, -0.415, 0]]
NR8: [axle, [0.352, -0.303, 0]]
CL1: [axle, [0.586, 0.261, 0]]
CL2: [axle, [0.595, 0.632, 0]]
CL3: [axle, [0.618, 0.251, 0]]
CL4: [axle, [0.637, 0.662, 0]]
CR1: [axle, [0.565, -0.253, 0]]
CR2: [axle, [0.567, -0.607, 0]]
CR3: [axle, [0.610, -0.262, 0]]
CR4: [axle, [0.611, -0.641, 0]]
FL1: [axle, [0.781, 0.718, 0]]
FL2: [axle, [0.763, 0.253, 0]]
FL3: [axle, [0.863, 0.192, 0]]
FL4: [axle, [1.185, 0.172, 0]]
FL5: [axle, [0.842, 0.718, 0]]
FL6: [axle, [0.875, 0.271, 0]]
FL7: [axle, [0.879, 0.234, 0]]
FL8: [axle, [1.180, 0.209, 0]]
FC1: [axle, [0.823, 0.0162, 0]]
FC2: [axle, [1.172, 0.00117, 0]]
FC3: [axle, [0.845, -0.0100, 0]]
FC4: [axle, [1.215, -0.0181, 0]]
FR1: [axle, [0.764, -0.695, 0]]
FR2: [axle, [0.768, -0.263, 0]]
FR3: [axle, [0.810, -0.202, 0]]
FR4: [axle, [1.203, -0.196, 0]]
FR5: [axle, [0.795, -0.702, 0]]
FR6: [axle, [0.803, -0.291, 0]]
FR7: [axle, [0.832, -0.240, 0]]
FR8: [axle, [1.210, -0.245, 0]]
segments:
- points: [NL1, NL2]
color: white
- points: [NL3, NL4]
color: white
- points: [NL5, NL6]
color: white
- points: [NL7, NL8]
color: white
- points: [NC1, NC2]
color: yellow
- points: [NC3, NC4]
color: yellow
- points: [NR1, NR2]
color: white
- points: [NR3, NR4]
color: white
- points: [NR5, NR6]
color: white
- points: [NR7, NR8]
color: white
- points: [CL1, CL2]
color: yellow
- points: [CL3, CL4]
color: yellow
- points: [CR1, CR2]
color: yellow
- points: [CR3, CR4]
color: yellow
- points: [FL1, FL2]
color: white
- points: [FL3, FL4]
color: white
- points: [FL5, FL6]
color: white
- points: [FL7, FL8]
color: white
- points: [FC1, FC2]
color: yellow
- points: [FC3, FC4]
color: yellow
- points: [FR1, FR2]
color: white
- points: [FR3, FR4]
color: white
- points: [FR5, FR6]
color: white
- points: [FR7, FR8]
color: white
Start by using the file hud.yaml. To visualize it, you do not need the
calibration data. It will be helpful to make sure that you can do the easy
parts of the exercise: loading the map, and drawing the lines.
To load a map file, use the function load_map provided in duckietown_utils:
from duckietown_utils import load_map
map_data = load_map(map_filename)
(Note that map is a reserved symbol name in Python.)
To load the intrinsic calibration parameters, use the function load_camera_intrinsics provided in duckietown_utils:
from duckietown_utils import load_camera_intrinsics
intrinsics = load_camera_intrinsics(robot_name)
To load the extrinsic calibration parameters (i.e. ground projection), use the function load_homography provided in duckietown_utils:
from duckietown_utils import load_homography
H = load_homography(robot_name)
From a file name like "/path/to/map1.yaml", you can obtain the basename without extension yaml by using the function get_base_name provided in duckietown_utils:
from duckietown_utils import get_base_name
filename = "/path/to/map1.yaml"
map_name = get_base_name(filename) # = "map1"
To remove the distortion from an image, use the function rectify provided in duckietown_utils:
from duckietown_utils import rectify
rectified_image = rectify(image, intrinsics)
To draw the line segments specified in a map file, use the render_segments method defined in the Augmenter class:
class Augmenter():
# ...
def ground2pixel(self):
'''Method that transforms ground points
to pixel coordinates'''
# Your code goes here.
return "???"
image_with_drawn_segments = augmenter.render_segments(image)
In order for render_segments to draw segments on an image, you must first implement the method ground2pixel.
Jonathan Michaux, Liam Paull, and Miguel de la Iglesia
During the lectures, we have discussed general filtering techniques, and specifically the histogram filtering approach that we are using to estimate our location within a lane in Duckietown.
This is a 2-dimensional filter over $d$ and $\theta$, the lateral displacement in the lane and the robot heading relative to the direction of the lane.
In this exercise, we will replace the histrogram filter with a particle filter.
Create a ROS node and package that takes as input the list of line segments detected by the line detector, and outputs an estimate of the robot position within the lane to be used by the lane controller. You should be able to run:
duckiebot $ source DUCKIETOWN_ROOT/environment.sh
duckiebot $ source DUCKIETOWN_ROOT/set_vehicle.name.sh
duckiebot $ roslaunch dt_filtering_ROBOT_NAME lane_following.launch
and then follow the instructions in Unit M-24 - Checkoff: Navigation for trying the lane following demo.
You should definitely look at the existing histogram filter for inspiration.
You may find this a useful resource to get started.
While you are working on your node and it is crashing, you need not kill and relaunch the entire stack (or even launch on your robot). You should build a workflow whereby you can quickly launch only the node you are developing from your laptop.
Submit the code using location-specific instructions
Liam Paull
During the lectures, we have discussed general filtering techniques, and specifically the histogram filtering approach that we are using to estimate our location within a lane in Duckietown.
This is a 2-dimensional filter over $d$ and $\theta$, the lateral displacement in the lane and the robot heading relative to the direction of the lane.
In this exercise, we will replace the histrogram filter with an Extended Kalman Filter.
Create a ROS node and package that takes as input the list of line segments detected by the line detector, and outputs an estimate of the robot position within the lane to be used by the lane controller. You should be able to run:
duckiebot $ source DUCKIETOWN_ROOT/environment.sh
duckiebot $ source DUCKIETOWN_ROOT/set_vehicle.name.sh
duckiebot $ roslaunch dt_filtering_ROBOT_NAME lane_following.launch
and then follow the instructions in Unit M-24 - Checkoff: Navigation for trying the lane following demo.
You should definitely look at the existing histogram filter for inspiration.
You may find this a useful resource to get started.
While you are working on your node and it is crashing, you need not kill and relaunch the entire stack (or even launch on your robot). You should build a workflow whereby you can quickly launch only the node you are developing from your laptop.
Submit the code using location-specific instructions
move here the exercises we had last year about branching.
We have a useful API that makes it easy to create programs with command line arguments.
to write
Create a program
$ bag-bounce --fps fps --speed speed
that shows a bouncing ball on the screen, as if it were a billiard
Write an implementation of bag-mark-spots.
bag-mark-spotsCreate a program that for each image, finds the pixels that are closest to a certain color, and creates as the output a big red, yellow white spot on them.
$ bag-mark-spots --input bag in --mark "[[255,0,0],255,0,0]," --size 5 --output [bag out]
Abstract the analysis above in a way that the same analysis code can be run equally from a bag or on the laptop.
Make a ROS node and two launch files:
Do the above using our API for filters.
We define the interface InstagramFilter and the EasyAlgo configuration files.
pointer to EasyAlgo
Write a node using the EasyNode framework that decides which filters to run given the configuration.
Also introduce the DUCKIETOWN_CONFIG_SEQUENCE.
Make them run our code, and also visualize whatβs going on
We use the following interface:
class FeatureDescription():
def feature(self, image):
""" returns a "feature description" """
def feature_compare(feature1, feature2):
pass
We also provide a basic skeleton.
Simplest feature: average color.
class AverageColor(FeatureDescription):
def feature(self, image):
return np.mean(image)
def feature_mismatch(f1, f2)
return np.abs(f1-f2)
Compute the similarity matrix for the
$ similarity --feature average_color --input input.bag --output dirname
to write
We introduce the support for parallel processing that we have in the APIs.
to write
to write
to write
to write
to write
to write
to write
to write
A good exercise is writing image-ops-tester yourself.
However, we already gave you a copy of image-ops-tester, which you
used in the previous step, so there is not much of a challenge. So, letβs go one level up, and considerβ¦
image-ops-tester-tester specificationWrite a program image-ops-tester-tester that tests whether
a program conforms to the specification of a image-ops-tester
given in Section 2.5 - Testing it works with image-ops-tester.
The image-ops-tester-tester program is called as follows:
$ image-ops-tester-tester candidate-image-ops-tester
This must return:
image-ops-tester-tester-testerWe provide you with a helpful program called image-ops-tester-tester-tester
that makes sure that a candidate script conforms to the specification of
an image-ops-tester-tester.
Use it as follows:
$ image-ops-tester-tester-tester candidate image-ops-tester-tester
This should return 0 if everything is ok, or different than 0 otherwise.
Even if things are tested, you can never be sure that the tests themselves work.
This part describes things that you should know about UNIX/Linux environments.
Documentation writers: please make sure that every command used has a section in these chapters.
apt installto write
apt updateto write
apt dist-upgradehold back packages
apt-keyto write
apt-markto write
apt-getto write
add-apt-repositoryto write
wajigto write
dpigsto write
Andrea
manThis is an interface to the on-line reference manuals. Whenever you meet some unfamiliar commands, try use man certain_command before Googling. You will find it extremely clear, useful and self-contained.
cdGo to the directory you want. If you just use:
laptop $ cd
then you will go to your home directory, i.e., ~
sudoWhenever you want to modify system files, you will need sudo. Commonly touched system files including /etc, /opt and so on. Since most of you have followed the guideline to use passwordless sudo, I would recommend that make sure what you are doing with sudo before you execute the command, otherwise you may need to reinstall the system.
lsList all the files and documents in the current directory. From ~/.bashrc, we know some commonly used alias. See more by man ls.
β la for ls -a which will list out all files and documents including the hidden ones(whose name starts with a dot).
β ll for ls -l which will display Unix file types, permissions, number of hard links, owner, group, size, last-modified date and filename.
cpcp fileA directoryB will copy the file A to directory B. See more by executing man cp.
mkdirMake new directory. See more by man mkdir.
touchUpdate the access and modification times of the input file to current time. See more by man touch.
rebootThis command must be executed as root. sudo required. This will reboot your laptop or Raspberry Pi. See more by man reboot.
shutdownThis command requires sudo. You can set a countdown to shutdown you machine. More by man shutdown.
rmRemove certain file. rm -r will remove files. More in man rm.
passwdUpdate password of the current user. Old password needed.
catCat some file will return you the content. More in man cat.
teeRead from standard input and write to standard output and files. More on man tee.
truncateto write
fdiskto write
mountto write
umountto write
losetupto write
gpartedto write
ddto write
syncto write
dfto write
to write
visudoto write
update-alternativesto write
udevadmto write
systemctlto write
maketo write
virtualenvto write
pipto write
raspi-configto write
vcgencmdto write
raspistillto write
jstestto write
swaponto write
mkswapto write
chmodto write
groupsto write
adduserto write
useraddto write
curlto write
wgetto write
sha256sumto write
xzto write
sourceYou can only do source file_name if the file can be executed by bash.
whichTell you the /bin/ directory of your command. This is useful to distinguish which python you are using if you have virtualenv.
exportto write
pgrepto write
npmto write
nodejsto write
ntpdateto write
chshto write
echoto write
shto write
fc-cacheto write
First plug in the USB drive nothing will work if you donβt do that first. Now ssh into your robot. On the command line type:
duckiebot $ lsusb
you should see your Sandisk USB drive as an entry. Congrats, you correctly plugged it in
duckiebot $ lsblk
Under name you should see sda1, with size about 28.7GB and nothing under the MOUNTPOINT column (if you see something under MOUNTPOINT congrats you are done.
Next make a directory to mount to:
duckiebot $ sudo mkdir /media/logs
Next mount the drive
duckiebot $ sudo mount -t vfat /dev/sda1 /media/logs -o umask=000
Test by running lsblk again and you should now see /media/logs under MOUNTPOINT
duckiebot $ sudo umount /media/logs
htopYou can use htop to monitor CPU usage.
$ sudo apt install htop
to write
iotopInstall using:
$ sudo apt install iotop
to write
Test SD Card (or any disk) speed using the following commands,
which write to a file called filename.
$ dd if=/dev/zero of=filename bs=500K count=1024
$ sync
$ echo 3 | sudo tee /proc/sys/vm/drop_caches
$ dd if=filename of=/dev/null bs=500K count=1024
$ rm filename
Note the sync and the echo command are very important.
Example results:
524288000 bytes (524 MB, 500 MiB) copied, 30.2087 s, 17.4 MB/s
524288000 bytes (524 MB, 500 MiB) copied, 23.3568 s, 22.4 MB/s
That is write 17.4 MB/s, read 22 MB/s.
A blank SD card.
An image file to burn.
An Ubuntu computer with an SD reader.
A burned image.
First, find out what is the device name for the SD card.
Insert the SD Card in the slot.
Run the command:
$ sudo fdisk -l
Find your device name, by looking at the sizes.
For example, the output might contain:
Disk /dev/mmcblk0: 14.9 GiB, 15931539456 bytes, 31116288 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
In this case, the device is /dev/mmcblk0. That will be the device
in the next commands.
You may see /dev/mmcblk0pX or a couple of similar entries for each partition on the card,
where X is the partition number. If you donβt see anything like that, take out
the SD card and run the command again and see what disappeared.
Before proceeding, unmount all partitions.
Before unmounting partitions, make sure you found the correct device in the previous
step. In particular, /dev/sda/ is the hard drive of your computer, and if you unmount its partitions,
you can erase important data, including important files and the operating system you are not currently
using.
Run df -h. If there are partitions like /dev/mmcblk0pn, then unmount
each of them. For example:
laptop $ sudo umount /dev/mmcblk0p1
laptop $ sudo umount /dev/mmcblk0p2
Now that you know that the device is device,
you can burn the image to disk.
Let the image file be image file.
Burn the image using the command dd:
laptop $ sudo dd of=device if=image file status=progress bs=4M
Use the name of the device, without partitions. i.e., /dev/mmcblk0, not
/dev/mmcblk0pX.
dd comand with status=progress parameter only work for dd βversion 8.24 ubuntu16.04.2
An image file to burn.
An Ubuntu computer.
A shrunk image.
Majority of content taken from here
We are going to use the tool gparted so make sure itβs installed
laptop $ sudo apt install gparted
Let the image file be image file and its name be imagename.
Run the command:
laptop $ sudo fdisk -l image file
It should give you something like:
Device Boot Start End Sectors Size Id Type
duckiebot-RPI3-LP-aug15.img1 2048 131071 129024 63M c W95 FAT32 (LBA)
duckiebot-RPI3-LP-aug15.img2 131072 21219327 21088256 10.1G 83 Linux
Take note of the start of the Linux partition (in our case 131072), letβs call it start.
Now we are going to mount the Linux partition from the image:
laptop $ sudo losetup /dev/loop0 imagename.img -o $((start*512))
and then run gparted:
laptop $ sudo gparted /dev/loop0
In gparted click on the partition and click βResizeβ under the βPartitionβ menu. Resize drag the arrow or enter a size
that is equal to the minimum size plus 20MB
This didnβt work well for me - I had to add much more than 20MB for it to work.
Click the βApplyβ check mark. Before closing the final screen click through the arrows in the dialogue box
to find a line such a βresize2fs -p /dev/loop0 1410048Kβ. Take note of the new size of your partition. Letβs
call it new size.
Now remove the loopback on the second partition and setup a loopback on the whole image and run fdisk:
laptop $ sudo losetup -d /dev/loop0
laptop $ sudo losetup /dev/loop0 image file
laptop $ sudo fdisk /dev/loop0
Command (m for help): enter d
Partition number (1,2, default 2): enter 2
Command (m for help): enter n
Partition type
p primary (1 primary, 0 extended, 3 free)
e extended (container for logical partitions)
Select (default p): enter p
Partition number (2-4, default 2): enter 2
First sector (131072-62521343, default 131072): start
Last sector, +sectors or +size{K,M,G,T,P} (131072-62521343, default 62521343): +new sizeK
on the last line include the + and the K as part of the size.
Created a new partition 2 of type 'Linux' and of size 10.1 GiB.
Command (m for help): enter w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument
The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).
Disregard the final error.
You partition has now been resized and the partition table has been updated. Now we will remove the loopback and then truncate the end of the image file:
laptop $ fdisk -l /dev/loop0
Device Boot Start End Sectors Size Id Type
/dev/loop0p1 2048 131071 129024 63M c W95 FAT32 (LBA)
/dev/loop0p2 131072 21219327 21088256 10.1G 83 Linux
Note down the end of the second partition (in this case 21219327). Call this end.
laptop $ sudo losetup -d /dev/loop0
laptop $ sudo truncate -s $(((end+1)*512)) image file
You now have a shrunken image file.
It might be useful to compress it, before distribution:
laptop $ xz image file
Andrea
Preliminary reading:
Basics of networking, including
.local names workXXX (ref to find).
to write
Make sure that you know:
hostnameto write
ping: are you there?to write
ifconfigto write
$ ifconfig
Andrea
to write
This installs the client:
$ sudo apt install ssh
This installs the server:
to write
This enables the server:
to write
The SSH configuration as a client is in the file
~/.ssh/config
Create the directory with the right permissions:
$ mkdir ~/.ssh
$ chmod 0700 ~/.ssh
Edit the file:
~/.ssh/config
(We suggest you use VIM to edit files; see a tutorial here.)
laptop or duckiebot? - LP
Then add the following lines:
HostKeyAlgorithms ssh-rsa
The reason is that Paramiko, used by roslaunch,
does not support the ECSDA keys.
To log in to a remote computer remote with user remote-user, use:
$ ssh remote-user@remote
Symptom: βOffending key errorβ.
If you get something like this:
Warning: the ECDSA host key for ... differs from the key for the IP address '... '
Offending key for IP in /home/user/.ssh/known_hosts:line
then remove line line in ~/.ssh/known_hosts.
This is a step that you will repeat twice: once on the Duckiebot, and once on your laptop.
The program will prompt you for the filename on which to save the file.
Use the convention
/home/username/.ssh/username@host name
/home/username/.ssh/username@host name.pub
where:
username is the current user name that you are using (ubuntu or your chosen one);host name is the name of the host (the Duckiebot or laptop);An SSH key can be generated with the command:
$ ssh-keygen -h
The session output will look something like this:
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
At this point, tell it to choose this file:
/home/username/.ssh/username@host name
Then:
Enter passphrase (empty for no passphrase):
Press enter; you want an empty passphrase.
Enter same passphrase again:
Press enter.
Your identification has been saved in /home/username/.ssh/username@host name
Your public key has been saved in /home/username/.ssh/username@host name.pub
The key fingerprint is:
XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX username@host name
The key's randomart image is:
+--[ RSA 2048]----+
| . |
| o o . |
| o = o . o |
| B . . * o|
| S o O |
| o o . E|
| o o o |
| o + |
| .. . |
+-----------------+
Note that the program created two files.
The file that contains the private key is
/home/username/.ssh/username@host name
The file that contains the public key has extension .pub:
/home/username/.ssh/username@host name.pub
Next, tell SSH that you want to use this key.
Make sure that the file ~/.ssh/config exists:
$ touch ~/.ssh/config
Add a line containing
IdentityFile ~/.ssh/PRIVATE_KEY_FILE
(using the filename for the private key).
make sure to include the full path to the file, not just the filename.
Check that the config file is correct:
$ cat ~/.ssh/config
...
IdentityFile ~/.ssh/PRIVATE_KEY_FILE
...
To copy the generated SSH key to the clipboard xclip can be used (Installation of xclip if necessary).
$ sudo apt-get install xclip
$ xclip -sel clip < ~/.ssh/username@host name.pub
You have two computers, called βlocalβ and βremoteβ,
with users βlocal-userβ and βremote-userβ. Here, we assume that local and remote are complete hostnames (such as duckiebot.local.).
The two computers are on the same network and they can ping each other.
You have created a keypair for local-user on local.
This procedure is described in Section 19.5 - Creating an SSH keypair.
From the local computer, local-user will be able to log in to
remote computer without a password.
First, connect the two computers to the same network, and make sure that you
can ping remote from local:
local $ ping remote.local
Do not continue if you cannot do this successfully.
If you have created a keypair for local-user, you will have a public key
in this file on the local computer:
/home/local-user/.ssh/local-user@local.pub
This file is in the form:
ssh-rsa long list of letters and numbers local-user@local
You will have to copy the contents of this file on the remote computer,
to tell it that this key is authorized.
On the local computer, run the command:
local $ ssh-copy-id remote-user@remote
now you should be able to login to the remote without a password:
local $ ssh remote-user@remote
This should succeed, and you should not be asked for a password.
Sometimes, SSH does not work because you have the wrong permissions on some files.
In doubt, these lines fix the permissions for your .ssh directory.
$ chmod 0700 ~/.ssh
$ chmod 0700 ~/.ssh/*
ssh-keygento write
iwconfigto write
iwlistWhat wireless networks do I have around?
$ sudo iwlist interface scan | grep SSID
Does the interface support 5 GHz channels?
$ sudo iwlist interface freq
Example output:
wlx74da38c9caa0 20 channels in total; available frequencies :
Channel 01 : 2.412 GHz
Channel 02 : 2.417 GHz
Channel 03 : 2.422 GHz
Channel 04 : 2.427 GHz
Channel 05 : 2.432 GHz
Channel 06 : 2.437 GHz
Channel 07 : 2.442 GHz
Channel 08 : 2.447 GHz
Channel 09 : 2.452 GHz
Channel 10 : 2.457 GHz
Channel 11 : 2.462 GHz
Channel 36 : 5.18 GHz
Channel 40 : 5.2 GHz
Channel 44 : 5.22 GHz
Channel 48 : 5.24 GHz
Channel 149 : 5.745 GHz
Channel 153 : 5.765 GHz
Channel 157 : 5.785 GHz
Channel 161 : 5.805 GHz
Channel 165 : 5.825 GHz
Current Frequency:2.437 GHz (Channel 6)
Note that in this example only some 5Ghz channels are supported (36, 40, 44, 48, 149, 153, 157, 161, 165); for example, channel 38, 42, 50 are not supported. This means that you need to set up the router not to use those channels.
scpto write
Use this command:
laptop $ scp hostname:/path/to/out.jpg .
to download out.jpg to your current directory.
rsyncto write
Andrea
To do quick changes to files, especially when logged remotely, we suggest you use the VI editor, or more precisely, VIM (βVI iMprovedβ).
Install like this:
$ sudo apt install vim
vito write
Suggested ~/.vimrc:
syntax on
set number
filetype plugin indent on
highlight Comment ctermfg=Gray
autocmd FileType python set complete isk+=.,(
to write
Use the > command to indent.
To indent 5 lines, use 5 > >.
To mark a block of lines and indent it, use V.
For example, use VJJ> to indent 3 lines.
Use < to dedent.
Following the instructions here:
$ sudo add-apt-repository ppa:webupd8team/atom
$ sudo apt update
$ sudo apt install atom
With Atom, you are able to remotely code on files located on your Duckiebot with a GUI. The benefit of using Atom is that you are able to install extensions such as an IDE for Python, a Markdown previewer, or just use custom themes to avoid coding in the terminal.
Follow these instructions:
Install remote-atom
laptop $ sudo apm install remote-atom
Now, we need to edit our SSH config so that any data send to the port 52698, which is the port remote-atom is using, is forwarded via SSH to our local machine. Edit the file β~/.ssh/configβ. There, you add βRemoteForward 52698 127.0.0.1:52698β to your host. The resulting host will look similar to
Host lex
User julien
Hostname lex.local
RemoteForward 52698 127.0.0.1:52698
Now, we need to connect to our duckiebot via SSH and install rmate (and simultaniously rename it to ratom)
duckiebot $ sudo wget -O /usr/local/bin/ratom https://raw.github.com/aurora/rmate/master/rmate
duckiebot $ sudo chmod +x /usr/local/bin/ratom
Now, you just need to launch Atom on your local machine, go to Packages->Remote Atom->Start Server.
You can now edit a file in a terminal connected to your duckiebot via SSH by typing
duckiebot $ sudo ratom filename
And atom will automatically open the file on your local machine. In the settings of remote-atom, you can also set the package to start the server automatically on launching atom on your local machine.
To be productive in coding, you need to have a proper IDE.
Look at how quickly you can create Python files by using an IDE (Figure 24.1). In this case, the editor is importing the required symbols. Think about how long would it take to do it without an IDE.
In addition to LiClipse, the one that is suggested for this class, there exist:
Follow the instructions at this page.
At the moment of writing, these are:
$ wget http://www.mediafire.com/file/rwc4bk3nthtxcvv/liclipse_4.1.1_linux.gtk.x86_64.tar.gz
$ tar xvzf liclipse_4.1.1_linux.gtk.x86_64.tar.gz
$ sudo ln -s `pwd`/liclipse/LiClipse /usr/bin/liclipse
Now you can run it using liclipse:
$ liclipse
When it runs for the first time, choose βuse this as defaultβ and click βlaunchβ.
Choose βImport-> General -> Existing project into workspaceβ. Select the folder ~/duckietown.
Only Import -> General -> Projects from Folder or Archive, selecting ~/duckuments worked for me. JT
This is not about the duckuments, itβs for duckietown - AC
If it asks about interpreters, select βauto configβ.
When it shows βuncheck settings that should not be changedβ, just click OK.
Go to βwindow/preferences/General/Keysβ.
Find βprintβ, and unbind it.
Find βQuick switch editorβ, and set it to Ctrl-P. (This is now the same as Atom.)
Find βPrevious tabβ and assign Ctrl-Shift-[ (This is now the same as Atom.)
Find βNext tabβ and assign Ctrl-Shift-]. (This is now the same as Atom.)
Find βShow in (PyDev package explorer)β and assign Ctrl-Shift-M.
Make sure that you can do the following tasks:
Use the global browser: Press Cmd-Shift-T, type βwhatβ.
It should autocomplete to what_the_duck. Press enter; it should jump to the file.
Switch among open editors with Ctrl-P.
Switch between tabs with Ctrl-Shift-], -[.
See the current file in the directory, using Cmd-Shift-M.
| On Linux | On Mac | |
| Ctrl-Shift-T | Cmd-Shift-T | Globals browser |
| Ctrl-P | Cmd-P | Quick editor switch |
| Ctrl-Shift-[ | Cmd-Shift-[ | Previous tab (needs to be configured) |
| Ctrl-Shift-] | Cmd-Shift-] | Next tab (needs to be configured) |
| Ctrl-Shift-M | Cmd-Shift-M | Show in (PyDev package explorer) |
| Ctrl-1 | Cmd-1 | Find symbol |
From the βPreferencesβ section, itβs suggested to:
Then, there is the issue of βcode completionβ. This is a love-it-or-hate-it issue. The choice is yours.
to write
Most importantly, please take the time to disable email notification for Slack.
The point of Slack is that you donβt get email. You can work on Duckietown only when you have the time to do so.
to write how
Also remove pop up notifications from the app. It should be a discrete notification that says βhey, when you have some time, look at Twistβ, not βHEY HEY HEY PAY ATTENTION TO ME NOWβ.
to write procedure
Andrea
You need to learn to use Byobu. It will save you much time later.
(Alternatives such as GNU Screen are fine as well.)
To write
On Ubuntu, install using:
$ sudo apt install byobu
See the screencast on the website http://byobu.co/.
You can change the escape sequence from Ctrl-A to something else by using the configuration tool that appears when you type F9.
Commands to use windows:
| Using function keys | Using escape sequences | |
| Create new window | F2 | Ctrl-A then C |
| Previous window | F3 | |
| Next window | F4 | |
| Switch to window | Ctrl-A then a number | |
| Close window | Ctrl-F6 | |
| Rename window | Ctrl-A then , |
Commands to use panes (windows split in two or more):
| Using function keys | Using escape sequences | |
| Split horizontally | Shift-F2 | Ctrl-A then | |
| Split vertically | Ctrl-F2 | Ctrl-A then % |
| Switch focus among panes | Ctrl-ββββ | Ctrl-A then one of ββββ |
| Break pane | Ctrl-A then ! |
Other commands:
| Using function keys | Using escape sequences | |
| Help | Ctrl-A then ? | |
| Detach | Ctrl-A then D |
Scroll up and down using fnoptionβ and fnoptionβ.
Highlight using alt
Andrea
The basic Git program is installed using
$ sudo apt install git
Additional utilities for git are installed using:
$ sudo apt install git-extras
This include the git-ignore utility.
This should be done twice, once on the laptop, and later, on the robot.
These options tell Git who you are:
$ git config --global user.email "email"
$ git config --global user.name "full name"
Also do this, and it doesnβt matter if you donβt know what it is:
$ git config --global push.default simple
Delete local:
$ git branch -d branch-name
Delete remote:
$ git push origin --delete branch-name
Propagate on other machines by doing:
$ git fetch --all --prune
You can clone without history with the command:
$ git clone --depth 1 repository URL
The symptom is:
$ git push
Username for 'https://github.com':
Diagnosis: the remote is not correct.
If you do git remote you get entries with https::
$ git remote -v
origin https://github.com/duckietown/Software.git (fetch)
origin https://github.com/duckietown/Software.git (push)
Expectation:
$ git remote -v
origin git@github.com:duckietown/Software.git (fetch)
origin git@github.com:duckietown/Software.git (push)
Solution:
$ git remote remove origin
$ git remote add origin git@github.com:duckietown/Software.git
git push complains about upstreamThe symptom is:
fatal: The current branch branch name has no upstream branch.
Solution:
$ git push --set-upstream origin branch name
gitto write
hubInstall hub using the instructions.
You can create a pull request using:
$ hub pull-request -m "description"
This describes Git LFS.
See instructions at:
Following these instructions, run the following:
$ sudo add-apt-repository ppa:git-core/ppa
$ curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
$ sudo apt update
$ sudo apt install git-lfs
unresolved issues.
The instructions above do not work.
Following this, the error that appears is that golang on the Pi is 1.6 instead it should be 1.7.
The binary files are not downloaded. In their place, there are short βpointerβ files.
If you have installed LFS after pulling the repository and you see only the pointer files, do:
$ git lfs pull --all
Andrea
This chapter describes how to create a Github account and setup SSH on the robot and on the laptop.
Our example account is the following:
Github name: greta-p
E-mail: greta-p@duckietown.com
Create a Github account (Figure 29.1).

Go to your inbox and verify the email.
Give the administrators your account name. They will invite you.
Accept the invitation to join the organization that you will find in your email.
You will do this procedure twice: once for the public key created on the laptop, and later with the public key created on the robot.
A public/private keypair already created and configured. This procedure is explained in Section 19.5 - Creating an SSH keypair.
You can access Github using the key provided.
Since I am not as familiar with Linux an VIM it would have been great to say how we can get access to the public key: sudo vim /home/username/.ssh/username@host name.pub and than copy it and add it on github SL
Go to settings (Figure 29.2).

Add the public key that you created:



To check that all of this works, use the command
$ ssh -T git@github.com
The command tries to connect to Github using the private keys that you specified. This is the expected output:
Warning: Permanently added the RSA host key for IP address 'ip address' to the list of known hosts.
Hi username! You've successfully authenticated, but GitHub does not provide shell access.
If you donβt see the greeting, stop.
Repeat what you just did for the Duckiebot on the laptop as well, making sure to change the name of the file containing the private key.
Liam
This part installs ROS. You will run this twice, once on the laptop, once on the robot.
The first commands are copied from this page.
Tell Ubuntu where to find ROS:
$ sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
Tell Ubuntu that you trust the ROS people (they are nice folks):
$ sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116
Fetch the ROS repo:
$ sudo apt update
Now install the mega-package ros-kinetic-desktop-full.
$ sudo apt install ros-kinetic-desktop-full
Thereβs more to install:
$ sudo apt install ros-kinetic-{tf-conversions,cv-bridge,image-transport,camera-info-manager,theora-image-transport,joy,image-proc,compressed-image-transport,phidgets-drivers,imu-complementary-filter,imu-filter-madgwick}
Do not install packages by the name of ros-X, only those by
the name of ros-kinetic-X. The packages ros-X are from another version of ROS.
XXX: not done in aug20 image:
Initialize ROS:
$ sudo rosdep init
$ rosdep update
rqt_consoleto write
roslaunchto write
rvizto write
rostopicto write
rostopic hzto write
rostopic echoto write
catkin_maketo write
rosrunto write
rostestto write
rospackto write
rosparamto write
rosdepto write
roswtfto write
rosbagA bag is a file format in ROS for storing ROS message data. Bags, so named
because of their .bag extension, have an important role in ROS.
Bags are typically created by a tool like
rosbag, which subscribe to one or
more ROS topics, and store the serialized message data in a file as it is received.
These bag files can also be played back in ROS to the same topics they were
recorded from, or even remapped to new topics.
rosbag recordThe command
rosbag record
records a bag file with the contents of specified topics.
rosbag infoThe command
rosbag info
summarizes the contents of a bag file.
rosbag playThe command
rosbag play
plays back the contents of one or more bag files.
rosbag checkThe command
rosbag check
determines whether a bag is playable in the current system, or if it can be migrated.
rosbag fixThe command
rosbag fix
repairs the messages in a bag file so that it can be played in the current system.
rosbag filterThe command
rosbag filter
converts a bag file using Python expressions.
rosbag compressThe command
rosbag compress
compresses one or more bag files.
rosbag decompressThe command
rosbag decompress
decompresses one or more bag files.
rosbag reindexThe command
rosbag reindex
re-indexes one or more broken bag files.
roscoreto write
computer is not in your SSH known_hosts file
See this thread. Remove the known_hosts file and make sure you
have followed the instructions in Section 19.3 - Local configuration.
PyTorch is a Python deep learning library thatβs currently gaining a lot of traction, because itβs a lot easier to debug and prototype (compared to TensorFlow / Theano).
To install PyTorch on the Duckietbot you have to compile it from source, because there is no pro-compiled binary for ARMv7 / ARMhf available. This guide will walk you through the required steps.
First you need to install some additional packages. You might already have installed. If you do, thatβs not a problem.
sudo apt-get install libopenblas-dev cython libatlas-dev m4 libblas-dev
In your current shell add two flags for the compiler
export NO_CUDA=1 # this will disable CUDA components of PyTorch, because the little RaspberriPi doesn't have a GPU that supports CUDA
export NO_DISTRIBUTED=1 # no idea what this does, but it fixed a compilation bug for me
Then cd into a directory of your choice, like cd ~/Downloads or something like that and clone the PyTorch library.
git clone --recursive https://github.com/pytorch/pytorch
There was recently a bug in the ARM-relevant code that should now be fixed in the main Github branch, but just to make sure you have the most recent code:β¦
Change into the directory that you just cloned, and further into the following direcotries:
cd pytorch/torch/lib/ATen/
β¦and check that the file Scalar.h has the following code on line 16:
Scalar() : Scalar(int64_t(0))
If the line instead reads the following, please manually change the code to the above line:
Scalar() : Scalar(0L)
When I was compiling the library I ran out of SWAP space (which is 500MB by default). I was successful in compiling it with 2GB of SWAP space. Here is how you can increase the SWAP (only for compilation - later we will switch back to 500MB).
Create the swap file of 2GB
sudo dd if=/dev/zero of=/swap1 bs=1M count=2048
Make this empty file into a swap-compatible file
sudo mkswap /swap1
Then disable the old swap space and enable the new one
sudo nano /etc/fstab
This above command will open a text editor on your /etc/fstab file. The file should have this as the last line: /swap0 swap swap. In this line, please change the /swap0 to /swap1. Then save the file with CTRL+o and ENTER. Close the editor with CTRL+x.
Now your system knows about the new swap space, and it will change it upon reboot, but if you want to use it right now, without reboot, you can manually turn off and empty the old swap space and enable the new one:
sudo swapoff /swap0
sudo swapon /swap1
cd into the main directory, that you clones PyTorch into, in my case cd ~/Downloads/pytorch and start the compilation process:
python setup.py build
This shouldnβt create any errors but it took me about an hour. If it does throw some exceptions, please let me know.
When itβs done, you can install the pytorch package system-wide with
sudo python setup.py install
For some reason on my machine this caused recompilation of a few packages. So this might again take some time (but should be significantly less).
If all of the above went through without any issues, congratulations. :) You should now have a working PyTorch installation. You can try it out like this.
First you need to change out of the installation directory (this is important - otherwise you get a really weird error):
cd ~
Then run Python:
python
And on the Python interpreter try this:
import torch
a = torch.FloatTensor((2,2))
a.add_(3)
print (a)
β¦this should print something like this:
3 3
3 3
[torch.FloatTensor of size 2x2]
Now if you like having 2GB of SWAP space (additional RAM basically, but a lot slower than your built-in RAM), then you are done. The downside is that you might run out of space later on. If you want to revert back to your old 500MB swap file then do the following:
Open the /etc/fstab file in the editor:
sudo nano /etc/fstab
TODO
please change the /swap0 to /swap1. Then save the file with
Caffe and TensorFlow are popular deep learning libraries, and are supported by the Intel Neural Computing Stick (NCS).
Install some of the dependencies first. The last command βsudo pip install .....β will cause some time.
sudo apt-get install -y gfortran cython
sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler git
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install -y python-dev libgflags-dev libgoogle-glog-dev liblmdb-dev libatlas-base-dev python-skimage
sudo pip install pyzmq jsonschema pillow numpy scipy ipython jupyter pyyaml
Then, you need to clone the repo of caffe
cd
git clone https://github.com/BVLC/caffe
Before compile Caffe, you have to modify Makefile.config
cd caffe
cp Makefile.config.example Makefile.config
sudo vim Makefile.config
Then, change four lines from
'#'CPU_ONLY := 1
/usr/lib/python2.7/dist-packages/numpy/core/include
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
to
CPU_ONLY := 1
/usr/local/lib/python2.7/dist-packages/numpy/core/include
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/arm-linux-gnueabihf/hdf5/serial/
Next, you can start to compile caffe
make all
make test
make runtest
make pycaffe
If you didβt get any error above, congratulation on your success. Finally, please export pythonpath
sudo vim ~/.bashrc
export PYTHONPATH=/home/"$USER"/caffe/python:$PYTHONPATH
Now, we can confirm whether the installation is successful. Download AlexNet and run caffe time
cd ~/caffe/
python scripts/download_model_binary.py models/bvlc_alexnet
./build/tools/caffe time -model models/bvlc_alexnet/deploy.prototxt -weights models/bvlc_alexnet/bvlc_alexnet.caffemodel -iterations 10
And you can see the benchmark of AlexNet on Pi3 caffe.
First, update apt-get:
$ sudo apt-get update
For Bazel:
$ sudo apt-get install pkg-config zip g++ zlib1g-dev unzip
For Tensorflow: (NCSDK only support python 3+. I didnβt use mvNC on rpi3, so here I choose python 2.7)
(For Python 2.7)
$ sudo apt-get install python-pip python-numpy swig python-dev
$ sudo pip install wheel
(For Python 3.3+)
$ sudo apt-get install python3-pip python3-numpy swig python3-dev
$ sudo pip3 install wheel
To be able to take advantage of certain optimization flags:
$ sudo apt-get install gcc-4.8 g++-4.8
$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 100
$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 100
Make a directory that hold all the thing you need
$ mkdir tf
Download and extract bazel (here I choose 0.7.0):
$ cd ~/tf
$ wget https://github.com/bazelbuild/bazel/releases/download/0.7.0/bazel-0.7.0-dist.zip
$ unzip -d bazel bazel-0.7.0-dist.zip
Modify some file:
$ cd bazel
$ sudo chmod u+w ./* -R
$ nano scripts/bootstrap/compile.sh
To line 117, add β-J-Xmx500Mβ:
$ nano tools/cpp/cc_configure.bzl
Place the line return βarmβ around line 133 (beginning of the _get_cpu_value function):
Build Bazel (it will take a while, about 1 hour):
$ ./compile.sh
When the build finishes:
$ sudo cp output/bazel /usr/local/bin/bazel
Run bazel check if itβs working:
$ bazel
Clone tensorflow repo (here I choose 1.4.0):
$ cd ~/tf
$ git clone -b r1.4 https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
(Incredibly important) Changes references of 64-bit program implementations (which we donβt have access to) to 32-bit implementations.
$ grep -Rl 'lib64' | xargs sed -i 's/lib64/lib/g'
Modify the file platform.h:
$ sed -i "s|#define IS_MOBILE_PLATFORM|//#define IS_MOBILE_PLATFORM|g" tensorflow/core/platform/platform.h
Configure the build: (important) if you want to build for Python 3, specify /usr/bin/python3 for Pythonβs location and /usr/local/lib/python3.x/dist-packages for the Python library path.
$ ./configure
Build the Tensorflow (this will take a LOOOONG time, about 7 hrs ):
$ bazel build -c opt --copt="-mfpu=neon-vfpv4" --copt="-funsafe-math-optimizations" --copt="-ftree-vectorize" --copt="-fomit-frame-pointer" --local_resources 1024,1.0,1.0 --verbose_failures tensorflow/tools/pip_package:build_pip_package
After finished compiling, install python wheel:
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
$ sudo pip install /tmp/tensorflow_pkg/tensorflow-1.4.0-cp27-none-linux_armv7l.whl
Check version:
$ python -c 'import tensorflow as tf; print(tf.__version__)'
And youβre done! You deserve a break.
Suppose you already have inception-v3 model (with inception-v3.meta and inception-v3.ckpt)
Create a testing python file
$ vim test.py
Write the following code:
Save, and excute it
$ python test.py cat.jpg
Then it will show the predict label and predict time.
This part is about how to develop software for the Duckiebot.
Install using:
$ sudo apt install virtualenv
matplotlib
seaborne
numpy
panda
scipy
opencv
...
to write
to write
how to catch and re-raise
to write
Use this recipe if you need to download things:
from duckietown_utils.download import download_if_not_exist
url = 'https://www.dropbox.com/s/bzezpw8ivlfu4b0/frame0002.jpg?dl=0'
f = 'local.jpg'
download_if_not_exist(url, f)
(Do not commit JPGs and other binary data to the Software repository.)
actually use urls.yaml
How to enter IPython:
from IPython import embed()
a = 10
embed() # enters interactive mode
segment_list = copy.deepcopy(segment_list)
YAML files are useful for writing configuration files, and are used a lot in Duckietown.
to write
to write
the yaml library
the ruamel.yaml library
to write
Never use tabs in Python files.
The tab characters are evil in Python code. Please be very careful in changing them.
Do not use a tool to do it (e.g. βConvert tabs to spacesβ); it will get it wrong.
checked by what-the-duck.
Indentation is 4 spaces.
Lines should be below 85 characters.
what-the-duck report those above 100 characters.
This is just a symptom of a bigger problem.
The problem here is that you do not do how to program well, therefore you create programs with longer lines.
Do not go and try to shorten the lines; the line length is just the symptom. Rather, ask somebody to take a look at the code and tell you how to make it better.
All files must have an encoding declared, and this encoding must be utf-8:
# -*- coding: utf-8 -*-
Executable files start with:
#!/usr/bin/env python
Comments refer to the next line.
Comments, bad:
from std_msgs.msg import String # This is my long comment
Comments, better:
# This is my long comment
from std_msgs.msg import String
For logging, import this logger:
from duckietown_utils import logger
DTConfigException
raise_wrapped
compact = True
def summary():
fs = get_all_configuration_files()
if __name__ == '__main__':
wrap_script_entry_point(summary)
Do not do a star import, like the following:
from rostest_example.Quacker import *
This chapter explains what are the assumptions about the configuration.
While the βSetupβ parts are βimperativeβ (do this, do that); this is the βdeclarativeβ part, which explains what are the properties of a correct configuration (but it does not explain how to get there).
The tool what-the-duck (Subsection 8.1.3 - The what-the-duck program) checks some of these conditions.
If you make a change from the existing conditions, make sure that it gets
implemented in what-the-duck by filing an issue.
You need to have set up the variables in Table 4.1.
The way to set these up is to add them in the file ~/.bashrc (export var="value"). Do not modify the environment.sh script.
| variable | reasonable value | contains |
DUCKIETOWN_ROOT |
~/duckietown |
Software repository |
DUCKIEFLEET_ROOT |
~/duckiefleet |
Where to look for class-specific information (people DB, robots DB). |
DUCKIETOWN_DATA |
~/duckietown-data |
The place where to look for logs. |
DUCKIETOWN_TMP |
If set, directory to use for temporary files. If not set, we use the default ( /tmp). |
|
DUCKIETOWN_CONFIG_SEQUENCE |
defaults:baseline:vehicle:user |
The configuration sequence for EasyNode |
DUCKIETOWN_ROOTto write
DUCKIEFLEET_ROOTFor Fall 2017, this is the the repository duckiefleet.
For self-guided learners, this is an arbitrary repository to create.
The system needs to know certain details about the robots, such as their host names, the name of the users, etc.
This data is contained in the ${DUCKIEFLEET_ROOT}/robots/{your_branch} directory,
in files with the pattern robot name.robot.yaml.
The file must contain YAML entries of the type:
owner: ID of owner
username: username on the machine
hostname: host name
description: generic description
log:
date: comment
date: comment
A minimal example is in Listing 4.6.
owner: censi
hostname: emma
username: andrea
description: Andrea's car.
log:
2017-08-01: >
Switched RPI2 with RPI3.
2017-08-20: >
There is something wrong with the PWM hat and the LEDs.
emma.robot.yamlExplanations of the fields:
hostname: the host name. This is normally the same as the robot name.username: the name of the Linux user on the robot, from which to run programs.owner: the ownerβs globally-unique Duckietown ID.machines fileMake sure you already set up ROS (Section 11.3 - Set up the ROS environment on the Duckiebot).
Activate ROS:
$ cd ~/duckietown
$ source environment.sh
The machines file is created from the scuderia data using this command:
$ rosrun duckieteam create-machines
Andrea
Describe the people database.
This is a globally-unique ID for people in the Duckietown project.
It is equal to the Slack username.
There is no Slack username anymore, so we should change this to some other convention. -AC
There are 3 modes of operation:
MODE-normal: Everything runs on the robot.MODE-offload: Drivers run on the robot, but heavy computation runs on the laptop.MODE-bag: The data is provided from a bag file, and computation runs on the laptop.| mode name | who is the ROS master | where data comes from | where heavy computation happen |
MODE-normal |
duckiebot |
Drivers on Duckiebot | duckiebot |
MODE-offload |
duckiebot |
Drivers on Duckiebot | laptop |
MODE-bag |
laptop |
Bag file | laptop |
Where the config files are, how they are used.
pkg_nameAndrea
This document outline the process of writing a ROS package and nodes in Python.
To follow along, it is recommend that you duplicate the pkg_name folder and edit the content of the files to make your own package.
CMakeLists.txtWe start with the file CMakeLists.txt.
Every ROS package needs a file CMakeLists.txt, even if you are just using Python code in your package.
documentation about CMakeLists.txt XXX
For a Python package, you only have to pay attention to the following parts.
The line:
project(pkg_name)
defines the name of the project.
The find_package lines:
find_package(catkin REQUIRED COMPONENTS
roscpp
rospy
duckietown_msgs # Every duckietown packages must use this.
std_msgs
)
You will have to specify the packages on which your package is dependent.
In Duckietown, most packages depend on duckietown_msgs to make use of the customized messages.
The line:
catkin_python_setup()
tells catkin to setup Python-related stuff for this package.
package.xmlThe file package.xml defines the meta data of the package.
Catkin makes use of it to flush out the dependency tree and figures out the order of compiling.
Pay attention to the following parts.
<name> defines the name of the package. It has to match the project name in CMakeLists.txt.
<description> describes the package concisely.
<maintainer> provides information of the maintainer.
<build_depend> and <run_depend>. The catkin packages this package depends on. This usually match the find_package in CMakeLists.txt.
setup.pyThe file setup.py configures the Python modules in this package.
The part to pay attention to is
setup_args = generate_distutils_setup(
packages=['pkg_name'],
package_dir={'': 'include'},
)
The packages parameter is set to a list of strings of the name of the folders inside the include folder.
The convention is to set the folder name the same as the package name. Here itβs the include/pkg_name folder.
You should put ROS-independent and/or reusable module (for other packages) in the include/pkg_name folder.
Python files in this folder (for example, the util.py) will be available to scripts in the catkin workspace (this package and other packages too).
To use these modules from other packages, use:
from pkg_name.util import *
talker.pyLetβs look at src/talker.py as an example.
ROS nodes are put under the src folder and they have to be made executable to function properly.
You use chmod for this; see Section 11.1 - chmod.
Header:
#!/usr/bin/env python
import rospy
# Imports module. Not limited to modules in this package.
from pkg_name.util import HelloGoodbye
# Imports msg
from std_msgs.msg import String
The first line, #!/usr/bin/env python, specifies that the script is written in Python.
Every ROS node in Python must start with this line.
The line import rospy imports the rospy module necessary for all ROS nodes in Python.
The line from pkg_name.util import HelloGoodbye imports the class HelloGoodbye defined in the file pkg_name/util.py.
Note that you can also include modules provided by other packages, if you
specify the dependency in CMakeLists.txt and package.xml.
The line from std_msgs.msg import String imports the String message defined in the std_msgs package.
Note that you can use rosmsg show std_msgs/String
in a terminal to lookup the definition of String.msg.
This is the main file:
if __name__ == '__main__':
# Initialize the node with rospy
rospy.init_node('talker', anonymous=False)
# Create the NodeName object
node = Talker()
# Setup proper shutdown behavior
rospy.on_shutdown(node.on_shutdown)
# Keep it spinning to keep the node alive
rospy.spin()
The line rospy.init_node('talker', anonymous=False) initializes a node named talker.
Note that this name can be overwritten by a launch file. The launch file can also push this node down namespaces. If the anonymous argument is set to True then a random string of numbers will be append to the name of the node. Usually we donβt use anonymous nodes.
The line node = Talker() creates an instance of the Talker object. More details in the next section.
The line rospy.on_shutdown(node.on_shutdown) ensures that the node.on_shutdown will be called when the node is shutdown.
The line rospy.spin() blocks to keep the script alive. This makes sure the node stays alive and all the publication/subscriptions work correctly.
Talker classWe now discuss the Talker class in talker.py.
In the constructor, we have:
self.node_name = rospy.get_name()
saves the name of the node.
This allows to include the name of the node in printouts to make them more informative. For example:
rospy.loginfo("[%s] Initializing." % (self.node_name))
The line:
self.pub_topic_a = rospy.Publisher("~topic_a", String, queue_size=1)
defines a publisher which publishes a String message to the topic ~topic_a. Note that the ~ in the name of topic under the namespace of the node. More specifically, this will actually publishes to talker/topic_a instead of just topic_a. The queue_size is usually set to 1 on all publishers.
For more details see rospy overview: publisher and subscribers.
The line:
self.sub_topic_b = rospy.Subscriber("~topic_b", String, self.cbTopic)
defines a subscriber which expects a String message and subscribes to ~topic_b. The message will be handled by the self.cbTopic callback function. Note that similar to the publisher, the ~ in the topic name puts the topic under the namespace of the node. In this case the subscriber actually subscribes to the topic talker/topic_b.
It is strongly encouraged that a node always publishers and subscribes to topics under their node_name namespace. In other words, always put a ~ in front of the topic names when you defines a publisher or a subscriber. They can be easily remapped in a launch file. This makes the node more modular and minimizes the possibility of confusion and naming conflicts. See the launch file section for how remapping works.
The line
self.pub_timestep = self.setupParameter("~pub_timestep", 1.0)
Sets the value of self.pub_timestep to the value of the parameter ~pub_timestep. If the parameter doesnβt exist (not set in the launch file), then set it to the default value 1.0. The setupParameter function also writes the final value to the parameter server. This means that you can rosparam list in a terminal to check the actual values of parameters being set.
The line:
self.timer = rospy.Timer(rospy.Duration.from_sec(self.pub_timestep), self.cbTimer)
defines a timer that calls the self.cbTimer function every self.pub_timestep seconds.
Contents:
def cbTimer(self,event):
singer = HelloGoodbye()
# Simulate hearing something
msg = String()
msg.data = singer.sing("duckietown")
self.pub_topic_name.publish(msg)
Everyt ime the timer ticks, a message is generated and published.
Contents:
def cbTopic(self,msg):
rospy.loginfo("[%s] %s" %(self.node_name,msg.data))
Every time a message is published to ~topic_b, the cbTopic function is called. It simply prints the message using rospy.loginfo.
You should always write a launch file to launch a node. It also serves as a documentation on the I/O of the node.
Letβs take a look at launch/test.launch.
<launch>
<node name="talker" pkg="pkg_name" type="talker.py" output="screen">
<param name="~pub_timestep" value="0.5"/>
<remap from="~topic_b" to="~topic_a"/>
</node>
</launch>
For the <node>, the name specify the name of the node,
which overwrites rospy.init_node() in the __main__ of talker.py. The
pkg and type specify the package and the script of the node, in this case
itβs talker.py.
Donβt forget the .py in the end (and remember to make the file executable through chmod).
The output="screen" direct all the
rospy.loginfo to the screen, without this you wonβt see any printouts (useful
when you want to suppress a node thatβs too talkative.)
The <param> can be used to set the parameters. Here we set the ~pub_timestep to 0.5. Note that in this case this sets the value of talker/pub_timestep to 0.5.
The <remap> is used to remap the topic names. In this case we are replacing ~topic_b with ~topic_a so that the subscriber of the node actually listens to its own publisher. Replace the line with
<remap from="~topic_b" to="talker/topic_a"/>
will have the same effect. This is redundant in this case but very useful when you want to subscribe to a topic published by another node.
First of all, you have to catkin_make the package even if it only uses Python. catkin makes sure that the modules in the include folder and the messages are available to the whole workspace. You can do so by
$ cd ${DUCKIETOWN_ROOT}/catkin_ws
$ catkin_make
Ask ROS to re-index the packages so that you can auto-complete most things.
$ rospack profile
Now you can launch the node by the launch file.
$ roslaunch pkg_name test.launch
You should see something like this in the terminal:
... logging to /home/username/.ros/log/d4db7c80-b272-11e5-8800-5c514fb7f0ed/roslaunch-robot name-15961.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is 1GB.
started roslaunch server http://robot name.local:33925/
SUMMARY
========
PARAMETERS
* /rosdistro: $ROS_DISTRO
* /rosversion: 1.11.16
* /talker/pub_timestep: 0.5
NODES
/
talker (pkg_name/talker.py)
auto-starting new master
process[master]: started with pid [15973]
ROS_MASTER_URI=http://localhost:11311
setting /run_id to d4db7c80-b272-11e5-8800-5c514fb7f0ed
process[rosout-1]: started with pid [15986]
started core service [/rosout]
process[talker-2]: started with pid [15993]
[INFO] [WallTime: 1451864197.775356] [/talker] Initialzing.
[INFO] [WallTime: 1451864197.780158] [/talker] ~pub_timestep = 0.5
[INFO] [WallTime: 1451864197.780616] [/talker] Initialzed.
[INFO] [WallTime: 1451864198.281477] [/talker] Goodbye, duckietown.
[INFO] [WallTime: 1451864198.781445] [/talker] Hello, duckietown.
[INFO] [WallTime: 1451864199.281871] [/talker] Goodbye, duckietown.
[INFO] [WallTime: 1451864199.781486] [/talker] Hello, duckietown.
[INFO] [WallTime: 1451864200.281545] [/talker] Goodbye, duckietown.
[INFO] [WallTime: 1451864200.781453] [/talker] Goodbye, duckietown.
Open another terminal and run:
$ rostopic list
You should see
/rosout
/rosout_agg
/talker/topic_a
In the same terminal, run:
$ rosparam list
You should see the list of parameters, including /talker/pub_timestep.
You can see the parameters and the values of the talker node with
$ rosparam get /talker
You can register a parameter in the launch file such that it is added to the ROS parameter dictionary. This allows you to call rospy.get_param() on you parameter from talker.py.
Edit your launch file to look like this:
<launch>
<arg name="pub_timestep" default="0.5" />
<node name="talker" pkg="pkg_name" type="talker.py" output="screen">
<param name="~pub_timestep" value="$(arg pub_timestep)"/>
<remap from="~topic_b" to="~topic_a"/>
</node>
</launch>
Previously, you should have had the line <param name="~pub_timestep" value="0.5" /> inside of the node tags. This sets a parameter of value 0.5 to be called /talker/pub_timestep. (Remeber that the tilde prefixes the variable with the current namespace). By adding the line <arg name="pub_timestep" default="1" />, we are telling the program to look for a parameter on the command line called pub_timestep, and that if it doesnβt find one, to use the value one. Then, value=$(arg pub_timestep) retrieves the value set in the previous line.
Within talker.py, we can get the value of the inputted parameter. You should already have the line:
self.pub_timestep = self.setupParameter("~pub_timestep", 1.0)
This calles the talkerβs setupParameter method, which contains the line:
value = rospy.get_param(param_name,default_value)
Where ~pub_timestep is passed in as parapm_name and 1.0 is passed in as the default value. Now that we have edited the launch file to accept a command line argument, value should be the value which is given on the command line, rather than 0.5.
You can test that this works by calling roslaunch with the added parameter:
$ roslaunch pkg_name test.launch pub_timestep:=3
This should cause the time between messages to become three seconds.
The functions rosparam list and rosparam info [param] are useful in debugging issues with registering a parameter.
You should document the parameters and the publish/subscribe topic names of each node in your package. The user should not have to look at the source code to figure out how to use the nodes.
~. This makes sure that the IO of the node is crystal clear.<param>) and topics (using <remap>) with each node.Andrea
We use Makefiles to describe frequently-used commands.
The command
$ make all
displays the help for each command. This help is also included here as Section 7.3 - Makefile help.
There is one Makefile at the root of the Software repository, which
includes other makefiles in the directory Makefiles/:
Makefile
Makefiles/
Makefile.build.mk
Makefile.demos.mk
Makefile.docker.mk
Makefile.generate.mk
Makefile.hw_test.mk
Makefile.maintenance.mk
Makefile.openhouse.mk
Makefile.stats.mk
Makefile.test.mk
Each child Makefile is called
Makefile.section.mk
and it should contain only targets of the form section-name.
For example, Makefile.stats.mk contains the targets βstats, stats-easy_node, stats-easy_logsβ.
The target called section should provide an help for the section.
For example, when you run make build, you see:
### Building commands
Commands to build the software.
- `make build-machines` : Builds the machines file.
- `make build-machines-clean` : Removes the machines file.
- `make build-clean` : Clean everything.
The output should be valid Markdown, so that it can be included in this documentation.
These provide statistics about the data and the configuration.
make stats-easy_node: Prints summary of declared nots using the EasyNode frameworks.make stats-easy_logs: Prints summary of available logs.make stats-easy_algo: Prints summary of available algorithms.These commands run the unit tests.
make test-all: Run all the tests.
make test-circle: The tests to run in continuous integration. .
make test-catkin_tests: Run the ROS tests.make test-anti_instagram: Run the anti_instagram tests.make test-comptests: Run the comptests tests.make test-comptests-clean: Run the comptests tests.make test-comptests-collect-junit: Collects the JUnit results.make test-download-logs: Downloads the logs needed for the tests.Commands to build the software.
make build-catkin : Runs catkin_make.make build-catkin-parallel : Runs catkin_make, with 4 threads.make build-catkin-parallel-max : Runs catkin_make, with many threads.
make build-machines : Builds the machines file.
make build-machines-clean : Removes the machines file.
make build-clean : Clean everything.
For using Docker images
make docker-build: Creates the image.make docker-upload: Uploads the image.make docker-clean: Removes all local images.Generation of documentation
make generate-all: Generates everything.make generate-help: Generates help.make generate-easy_node: Generates the easy node documentation.make generate-easy_node-clean: Cleans the generated files.These are simple demos
to write
To perform hardware tests:
make hw-test-camera : Testing Camera HW by taking a picture (smile!).make hw-test-turn-right: Calibration right turnmake hw-test-turn-left: Calibrating left turnmake hw-test-turn-forward: Calibrating forward turnA couple of utilities for robot maintenance.
make maintenance-fix-time: Fixes the time.make maintenance-clean-pyc: Removes pyc files.These were the open house demos.
to write
Pull the branch 1710-place-recognition
$ cd duckietown
$ git pull
Source environment:
$ source environment.sh
Remove previous installations:
$ sudo apt remove ipython ipython-notebook
Then run:
$ pip install --user jupyter IPython==5.0
Check the versions are correct:
$ which ipython
/home/andrea/.local/bin/ipython
Check the version is correct:
$ ipython --version
5.0.0
Set a password:
$ jupyter notebook password
$ jupyter notebook --notebook-dir=$DUCKIETOWN_ROOT/catkin_ws/src/75-notebooks
Create a configuration file:
$ jupyter notebook --generate-config
Edit the file ~/.jupyter/jupyter_notebook_config.py.
Uncomment and change the line:
#c.NotebookApp.ip = 'localhost'
Into:
c.NotebookApp.ip = '*'
Andrea
This chapter describes formally what makes a conforming ROS package in the Duckietown software architecture.
β For exercises packages, the name of the package must be package_handle.
package.xmlβ There is a package.xml file.
Checked by what-the-duck.
β The messages are called β¦.
β There is a README.md file
Checked by what-the-duck.
β there is the first launch file
to write
This sections contains the documentation about the utility functions used for
image processing available in the duckietown_utils Python package.
write_image_as_jpgDescription: Takes an BGR image and writes it as a JPEG file.
Are we sure that the encoding is right? -AC
Prototype:
write_image_as_jpg( image, filename )
Defined in:
image_writing.py.
Arguments:
| Name | Type | Description |
| image | numpy.ndarray | The BGR image to save as JPEG file. |
| filename | str | The path of the JPEG file. |
Returns:
None.
rgb_from_rosDescription: Takes a ROS message containing an image and returns its RGB representation.
Prototype:
rgb_from_ros( msg )
Defined in:
image_conversions.py.
Arguments:
| Name | Type | Description |
| msg |
sensor_msgs.Image
or sensor_msgs.CompressedImage |
Message containing the image to extract. |
Returns:
numpy.ndarray
:: RGB representation of the image contained in the ROS message msg.
d8_compressed_image_from_cv_imageDescription: Takes a OpenCV image (BGR format), compresses it and wraps it into a ROS message of type
sensor_msgs.CompressedImage.
Prototype:
d8_compressed_image_from_cv_image( image_cv )
Defined in:
image_jpg_create.py.
Arguments:
| Name | Type | Description |
| image_cv | numpy.ndarray | BGR representation of the image to compress. |
Returns:
sensor_msgs.CompressedImage
:: A ROS message containing a compressed version of the input image_cv.
This unit describes how to debug your programs.
Do read this accurately top-to-bottom. If you think this is too long and too verbose to read and you are in a hurry anyway: that is probably the attitude that introduced the bug.
First, count your blessings. You are lucky to live in the present. Once, there were actual bugs in your computer (Figure 11.1).
The first truth is the following:
It is always something simple.
People tend to make up complicated stories in their head about what is happening. One reason they do that is because when you are frustrated, it is better to imagine to battle against an imaginary dragon, rather than a little invisible Leprechaun who is playing tricks on you.
Especially in an easy environment like Linux/ROS/Python with coarse process-level parallelization, there is really little space for weird bugs to creep in. If you were using parallel C++ code, you would see lots of heisenbugs). Here, the reason is always something simple.
The second truth is the following:
While there are bugs in the system, it is more likely there is a bug in your code or in your environment.
Any problem that has to do with libraries not importing, commands not existing, or similar, are because the environment is not set up correctly. Biggest culprit: forgetting βsource environment.shβ before doing anything, or rushing through the setup steps ignoring the things that failed.
Permission errors are most likely because people randomly used βsudoβ, thus creating root-owned files where they shouldnβt be.
Please report these, so that we can fix them.
Please report these, so that we can find workaround.
Make sure that you have pulled duckiefleet, and pushed your changes.
Finally, given the questions we had so far, I can give you the prior distribution of mistakes:
Of these, 80% is something that would be obvious by looking at the stack trace and your code and could be easily fixed.
If it is later than 10pm, just go to bed, and look at it tomorrow.
After 10pm, bugs are introduced, rather than removed.
Bug squashing requires a clear mind.
If you are in a hurry, itβs better you do this another time; otherwise, you will not find the bug and you will only grow more frustrated.
what-the-duckFinding a bug is a process of elimination of causes one-by-one, until you find the real culprit. Most of the problems come ultimately from the fact that your environment is not set up correctly.
We have a diagnostics program called what-the-duck that checks many things about the environment.
So, first of all, run what-the-duck. Then, fix the errors that what-the-duck shows you.
This is the proper way to run what-the-duck:
$ cd ~/duckietown
$ source environment.sh
$ git checkout master
$ git pull
$ ./dependencies_for_duckiebot.sh # if you are on a Duckiebot
$ ./dependencies_for_laptop.sh # if you are on a laptop
$ ./what-the-duck
you have to do all the steps in the precise order.
The tool also produces an HTML report about your system which you should attach to any request for help.
I notice many people just writing: βI get this error: β¦ How can I fix it?β. This is not the best way to get help. If you donβt include the code and stack trace, itβs hard to impossible to help you.
The best way to get help is the following:
Gold standard: Provide exact instructions on how to reproduce the error (βCheck out this branch; run this command; I expect this; instead I get thatβ). This makes it easy for an instructor or TA to debug your problem in 30 seconds, give you the fix, and probably fix it for everybody else if it is a common problem.
Silver standard: Copy the relevant code to a Gist (gist.github.com) including the error stack trace. Because we have no way to reproduce the error, this starts a conversation which is basically guesswork. So you get half answers after a few hours.
what-the-duckIf there are errors reported, the students should fix those before worrying about their current problem. Maybe you or they donβt see the connection, but the connection might be there.
Also, in general, errors in the environment will cause other problems later on.
The worst thing you can do is guess work β this causes confusion.
I encourage the TAs to not answer any nontrivial question that is not at least at the silver standard. It is a waste of resources, it will likely not help, and it actually contributes to the confusion, with people starting to try random things until something works without understanding why things work, and ultimately creating a culture of superstitions.
catkin_make -C catkin_ws/ --pkg easy_logs
These are the conventions for the Duckietown repositories.
The Software and the duckuments repository use βcontinuous integrationβ.
This means that there are well-defined tests that must pass at all times.
For the Software repository, the tests involve building the repository and running unit tests.
For the duckuments repository, the tests involve trying to build the documentation using make compile.
If the tests do not pass, then we say that we have βbroken the buildβ.
We also say that a branch is βgreenβ if the tests pass, or βredβ otherwise.
If you use the Chrome extension Pointless, you will see a green dot in different places on Github to signify the status of the build (Figure 13.1).

The system enforces the constraint that the branch master is always green,
by preventing changes to the branches that make the tests fail.
We use a service called CircleCI. This service continuously looks at our repositories. Whenever there is a change, it downloads the repositories and runs the tests.
(It was a lot of fun to set up all of this, but fortunately you do not need to know how it is done.)
At this page you can see the summary of the tests. (You need to be logged in with your Github account and click βauthorize Githubβ).
master: pull requestsIt is not possible to push on to the master branch directly.
See the Github documentation about pull requests to learn about the general concept.
The workflow is as follows.
(1) You make a private branch, say your name-devel.
(2) You work on your branch.
(3) You push often to your branch. Every time you push, CircleCI will run the tests and let you know if the tests are passing.
(4) When the tests pass, you create a βpull requestβ. You can do this by going to the Github page for your branch and click on the button βcompare and pull requestβ (Figure 13.3).

(5) You now have an opportunity to summarize all the changes you did so far (Figure 13.4). Then click βcreate pull requestβ.

(6) Now the pull request has been created. Other people can see and comment on it. However, it has not been merged yet.
At this point, it might be that it says βSome checks havenβt completed yetβ (Figure 13.5). Click βdetailsβ to see whatβs going on, or just wait.

When itβs done, you will see either a success message (Figure 13.6) or a failure message (Figure 13.7).


(7) At this point, you can click βsquash and mergeβ to merge the changes into master (Figure 13.8).

If you see a message like βmerge failedβ (Figure 13.9), it probably means that somebody pushed into master; merge master into your branch and continue the process.

You have implemented a new feature or improved an existing feature in your branch devel-project_name
A robot that is configured and able to follow lanes in Duckietown according to instructions in Unit M-24 - Checkoff: Navigation
Your branch gets merged into master and doesnβt break anything
This page is about what to do once you have developed and tested your new or improved feature and you want to contribute it back to the master branch.
If your branch is not merged into master and passes all the tests it will be lost forever.
Master into your branch$ git merge origin master
You should write some?
How to test that your branch is passing the tests? Look here ?
If you have developed a perception module you should define a set of regression tests that ensure that it is working properly.
TODO:
Once the simulator is up and running it should become part of the testing infrastructure.
If you have been doing your development on your laptop or some other computer, the time is now to put the code on the RasPI and test.
Verify that the previous functionality of the robot is preserved. For now repeat the instructions in Unit M-24 - Checkoff: Navigation, and ensure that the basic lane following and indefinite navigation abilities are preserved.
Once all of the tests are passed and you are sufficiently convinced that your code is not going to break the system, you should make a Pull Request by going here. For the base branch put master and for the compare branch put your branch.
This part describes the Duckietown algorithms and system architecture.
We do not go in the software details. The implementation details have been already talked about at length in Part K - Software development guide.
We do give links to the ROS packages implementing the functionality.
add video here
Drivers:
Operator interface:
to write
to write
to write
to write
video here
Perception:
anti_instagramground_projectionline_detector, Unit Q-6 - Package line_detector2lane_filterControl:
add video here
The packages involved in this functionality are:
we donβt discuss the details of the packages here; we just give pointers to them.
add video here
In the source code, a node must only publish/subscribe to private topics.
In rospy, this means that the topic argument of rospy.Publisher and rospy.Subscriber should always have a leading ~. ex: ~wheels_cmd, ~mode.
In roscpp, this means that the node handle should always be initialized as a private node handle by supplying with a "~" agrument at initialization. Note that the leading β~β must then be obmitted in the topic names of. ex:
ros::NodeHandle nh_("~");
sub_lineseglist_ = nh_.subscribe("lineseglist_in", 1, &GroundProjection::lineseglist_cb, this);
pub_lineseglist_ = nh_.advertise<duckietown_msgs::SegmentList> ("lineseglist_out", 1);
All the parameters of a node must be private parameters to that node.
All the nodes must write the value of the parameters being used to the parameter server at initialization. This ensures transparence of the parameters. Note that the get_param(name,default_value) does not write the default value to the parameter server automatically.
The default parameter of pkg_name/node_name should be put in ~/duckietown/catkin_ws/src/duckietown/config/baseline/pkg_name/node_name/dafault.yaml. The elemental launch file of this node should load the parameter using <rosparam>.
The above is deprecated. The configuration is handled differently.
Each node must have a launch file with the same name in the launch folder of the package. ex: joy_mapper.py must have a joy_mapper.launch. These are referred to as the elemental launch files.
Each elemental launch file must only launch one node.
The elemental launch file should put the node under the correct namespace through the veh arg, load the correct configuration and parameter file throught config and param_file_name args respectively. veh must not have a default value. This is to ensure the user to always provide the veh arg. config must be default to baseline and param_file_name must be default to default.
When a node can be run on the vehicle or on a laptop, the elemental launch file should provide a local arg. When set to true, the node must be launch on the launching machine, when set to false, the node must be launch on a vehicle throught the mahcine attribute.
A node should always be launched by calling its corresponding launch file instead of using rosrun. This ensures that the node is put under the correct namespace and all the necessary parameters are provided.
Do not use <remap> in the elemental launch files.
Do not use <param> in the elemental launch files.
Welcome to the Fall 2017 Duckietown experience.
This is the first time that a class is taught jointly across 3 continents!
There are 4 universities involved in the joint teaching for the term:
This part of the Duckiebook describes all the information that is needed by the students of the four institutions.
At ETHZ, UdeM, TTIC, the class will be more-or-less synchronized. The materials are the same; there is some slight variation in the ordering.
Moreover, there will be some common groups for the projects.
The NCTU class is undergraduate level. Students will learn slightly simplified materials. They will not collaborate directly with the other classes.
The first rule of Duckietown
The first rule of Duckietown is: you donβt talk about Duckietown, using email.
Instead, we use a communication platform called Slack.
There is one exception: inquiries about βmetaβ level issues, such as course enrollment and other official bureaucratic issues can be communicated via email.
The second rule of Duckietown
The second rule of Duckietown is: be kind and respectful, and have fun.
The third rule of Duckietown
The third rule of Duckietown is: read the instructions carefully.
Do not blindly copy and paste.
Only run a command if you know what it does.
Welcome aboard! We are so happy you are joining us at Duckietown!
This is your onboarding procedure. Please read all the steps and then complete all the steps.
If you do not follow the steps in order, you will suffer from unnecessary confusion.
If you donβt already have a Github account, sign up now.
Please use your full name when it asks you. Ideally, the username should be
something like FirstLast or something that resembles your name.
When you sign up, use your university email. This allows to claim an educational discount that will be useful later.
For ETHZ, UdeM and TTIC fill in this Preliminary Student Questionnaire.
Zurich: Please fill in questionnaire by Tuesday, September 26, 15:00 (extended from original deadline of 12:00).
if you have problems with this step, please contact Jacopo Tani <tanij@ethz.ch>.
After we receive the questionnaire, we will invite you to the Duckietown organization. You need to accept the invite; until you do, you are not part of the Duckietown organization and canβt access our repositories.
The invite should be waiting for you at this page.
After we receive the questionnaire, we will invite you to Slack.
The primary mode of online confabulation between staff and students is Slack, a team communication forum that allows the community to collaborate in making Duckietown awesome.
(Emails are otherwise forbidden, unless they relate to a private, university-based administrative concern.)
We will send you an invite to Slack. Check your inbox.
If after 24 hours from sending the questionnaire you havenβt received the invite, contact HR representative Kirsten Bowser <akbowser@gmail.com>.
What is Slack? More details about Slack are available here. In particular, remember to disable email notifications.
Slack username. When you accept your Slack invite, please identify yourself with first and last names followed by a β-β and your institution.
Andrea Censi - Zurich
Slack picture. Please add a picture (relatively professional, with duckie accessories encouraged).
Slack channels. A brief synopsis of all the help-related Slack channels is here: Unit M-10 - Slack Channels.
Check out all the channels in Slack, and add yourself to those that pertain or interest you. Be sure to introduce yourself in the General channel.
This is an optional step.
If you wish to connect with the Duckietown alumni network, on LinkedIn you can join the company βDuckietown Engineeringβ, with the title βVehicle Autonomy Engineer in trainingβ. Please keep updated your Linkedin profile with any promotions you might receive in the future.
If you do not have access to a laptop that meets the following requirements, please post a note in the channel #help-laptops.
You need a laptop with these specifications:
Linux Ubuntu 16.04 installed natively (dual boot), not in a virtual machine. See Subsection 2.3.1 - Can I use a virtual machine instead of dual booting? below for a discussion of the virtual machine option.
A WiFi interface that supports 5 GHz wireless networks. If you have a 2.4 GHz WiFi, you will not be able to comfortably stream images from the robot; moreover, you will need to adapt certain instructions.
Minimum 50 GB of free disk space in addition to the OS. Ideally you have 200 GB+. This is for storing and processing logs.
There are no requirements of having a particularly good GPU, or a particularly good CPU. You will be developing code that runs on a Raspberry PI. Any laptop bought in the last 3 years should be powerful enough. However, having a good CPU / lots of RAM makes it faster to run regression tests.
Running things in a virtual machine is possible, but not supported.
This means that while there is a way to make it work (in fact, Andrea develops in a VMWare virtual machine on OS X), we cannot guarantee that the instructions will work on a virtual machine, and, most importantly, the TAs will not help you debug those problems.
The issues that you will encounter are of two types.
There are performance issues. For example, 3D acceleration might not work in the virtual machine.
Most importantly, there are network configuration issues. These come up late in the class, when you start connecting the laptop to the Duckiebot. At that point, ROS makes certain assumptions about subnets, that might not be satisfied by your virtual machine configuration. At that point, you need to be relatively skilled to fix it.
So, the required skill here is not βbeing able to install Ubuntu on a virtual machineβ, but rather βBeing able to debug network problems involving multiple real/virtual networks and multiple real/virtual adaptersβ.
Hereβs a quiz: do these commands look familiar to you?
$ route add default gw 192.168.1.254 eth0
$ iptables -A FORWARD -o eth1 -j ACCEPT
If so, then things will probably work ok for you.
Otherwise, we strongly suggest that you use dual booting instead of a virtual machine.
At this point, you should be all set up, able to access our Github repositories, and, most important of all, able to ask for help on Slack.
You can now get acquainted to the class journal, to know the next steps.
Also, in this page, we will collect the logistics information (lab times, etc.).
To receive your Duckiebox on Wednesday Sep 27, you need to prove to be able to edit the Duckuments successfully.
See the instructions in this section.
If you canβt come on Wednesday, please contact one of the TAs.
to write
This section describes information specific to ZΓΌrich.
These are the local TAs:
Please contact them on Slack, rather than email.
Also feel free to contact the TAs in Montreal and Chicago.
Feel free to contact Ms. Kirsten Bowser (akbowser@gmail.com) if you have problems regarding accounts, permissions, etc.
During the term, we are not going to update the website.
Rather, all important information, such as deadlines, is in the class journal.
The point of contact for Duckiebox distribution is Shiying Li.
The local Duckietown room is ML J 44.2.
write opening hours and rules
double check the room number -AC
There will be extra lab space available.
Space-time coordinates TBD.
This unit contains all necessary info specific for students at UniveristΓ© de MontrΓ©al.
This is the official course website. It contains links to the the syllabus and description and other important info.
The authoritative class schedule will be tracked in Unit M-12 - MontrΓ©al branch diary. This will contain all lecture material, homeworks, checkoffs, and labs.
The lab room for the class is 2333 in Pavillion AndrΓ©-Aisenstadt. The code for the door is XXX. Please do not distribute the code for the door, we are trying to limit access to this room as much as possible.
The TA for the class is Florian Golemo. All communications with the course staff should happen through Slack.
The instructor is Prof. Liam Paull, whose office is 2347 Pavillion AndrΓ©-Aisenstadt.
It is preferable that you keep your robot for the semester. However, if you do not have a secure location where you can store it, we can store it for you in Room XXX in Pavillion AndrΓ©-Aisenstadt. However, you will have to ask Prof. Liam Paull to access or store your robot there each time since we cannot give out access to this space to the students in the class.
Matt
This section describes information specific to TTIC and UChicago students.
The course website provides a copy of the syllabus, grading information, and details on learning objectives.
Classes take place on Mondays and Wednesdays from 9am-11am in TTIC Room 530. In practice, each class will be divided into an initial lecture period (approximately one hour), followed by a lab session.
The class schedule is maintained as part of the TTIC Class Diary, which includes details on lecture topics, links to slides, etc.
The following is taken from the course syllabus:
The class will assess your grasp of the material through a combination of problem sets, exams, and a final project. The contribution of each to your overall grade is as follows:
See the course syllabus for more information on how the participation and final project grades are determined.
The following is taken from the course syllabus:
Late problem sets will be penalized 10% for each day that they are late. Those submitted more than three days beyond their due date will receive no credit.
Each student has a budget of three days that they can use to avoid late penalties. It is up to the student to decide when/how they use these days (i.e., all at once or individually). Students must identify whether and how many days they use when they submit an assignment.
It is not acceptable to use code or solutions from outside class (including those found online), unless the resources are specifically suggested as part of the problem set.
You are encouraged to collaborate through study groups and to discuss problem sets and the project in person and over Slack. However, you must acknowledge who you worked with on each problem set. You must write up and implement your own solutions and are not allowed to duplicate efforts. The correct approach is to discuss solution strategies, credit your collaborator, and write your solutions individually. Solutions that are too similar will be penalized.
Duckietown labs will take place at TTIC in the robotics lab on the 4th floor.
Note: TTIC and U. Chicago students in Matthew Walterβs research group use the lab as their exclusive research and office space. It also houses several robots and hardware to support them. Please respect the space when you use it: try not to distract lab members while they are working and please donβt touch the robots, sensors, or tools.
Duckietown is a collaborative effort involving close interaction among students, TAs, mentors, and faculty across several institutions. The local learning assistants (LAs) at TTIC are:
Nick and Eric (Nickβs student)
This section describes information specific to NCTU students.
The Duckietown Taiwan Branch Website provides some details about Duckietown Branch in NCTU-Taiwan and results of previous class in NCTU.
Classes take place on Thursday from 1:20pm~4:20om in NCTU Engineering Building 5 Room 635. Each class will be devided into two sessoins. In the first session, Professor Wang will give lessons on foundamental theory and inspire students to come up more creatitive but useful ideas on final projects. In the second sessoin, TAs will give pratical lab on how to use Duckietown platform as their project platform and use ROS as their middleware toward a fantastic work.
The class schedule is maintained as part of the NCTU Class Diary, which includes details on lecture topics, links to slides, etc.
The following is taken from the course syllabus:
This course aims at developing software projects usable in real-world, and focuses on βlearning by doing,β βteam work,β and βresearch/startup oriented.β. The contribution of each to your overall grade is as follows:
See the course syllabus for more information on course object and grading policy.
The following is taken from the course syllabus:
Late problem sets will be penalized 10% for each day that they are late. Those submitted more than three days beyond their due date will receive no credit.
Each student has a budget of three days that they can use to avoid late penalties. It is up to the student to decide when/how they use these days (i.e., all at once or individually). Students must identify whether and how many days they use when they submit an assignment.
It is not acceptable to use code or solutions from outside class (including those found online), unless the resources are specifically suggested as part of the problem set.
You are encouraged to collaborate through study groups and to discuss problem sets and the project in person and over Slack. However, you must acknowledge who you worked with on each problem set. You must write up and implement your own solutions and are not allowed to duplicate efforts. The correct approach is to discuss solution strategies, credit your collaborator, and write your solutions individually. Solutions that are too similar will be penalized.
Duckietown labs will take place at NCTU in the same place with the lecture.
Note: The course focus on βlearning by doingβ which means that the lab session of each class is expectially important.
Duckietown is a collaborative effort involving close interaction among students, TAs, mentors, and faculty across several institutions. The local learning assistants (LAs) at NCTU are:
There are some difference among the branches. These will be marked using the following graphical notation.
This is only for Zurich.
This is only for Montreal.
This is only for Chicago.
This is only for Taiwan.
These are the repositories we use.
The Software repository is the main repository that contains the software.
The URL to clone is:
git@github.com:duckietown/Software.git
In the documentation, this is referred to as DUCKIETOWN_ROOT.
During the first part of the class, you will only read from this repository.
The duckiefleet repository contains the data specific to this instance of the class.
The URL to clone is:
git@github.com:duckietown/duckiefleet.git
In the documentation, the location of this repo is referred to as DUCKIEFLEET_ROOT.
You will be asked to write to this repository, to update the robot DB and the people DB, and for doing exercises.
For homework submissions, we will use the following URL:
git@github.com:duckietown/exercises-fall2017.git
As explained below, it is important that this repo is kept separate so that students can privately work on their exercises at schools where the homeworks are counted for grades.
The Duckuments repository is the one that contains this documentation.
The URL to clone is:
git@github.com:duckietown/duckuments.git
Everybody is encouraged to edit this documentation!
In particular, feel free to insert comments.
The lectures repository contains the lecture slides.
The URL to clone is:
git@github.com:duckietown/lectures.git
Students are welcome to use this repository to get the slides, however, please note that this is a space full of drafts.
The exercises repository contains the solution to exercises.
The URL to clone is:
git@github.com:duckietown/XX-exercises.git
Only TAs have read and write permissions to this repository.
This does not apply to Zurich.
Homeworks will require you to write and submit coding exercises. They will be submitted using git. Since we have a university plagiarism policy (UdeMβs, TTIC/UChicago) we have to protect students work before the deadline of the homeworks. For this reason we will follow these steps for homework submission:
If you have not yet cloned the duckietown repo do it now:
$ git clone git@github.com:duckietown/exercises-fall2017.git
Now you need to point the remote of your exercises-fall2017 to your new local private repo. To do, from inside your already previously cloned exercises-fall2017 repo do:
$ git remote set-url origin git@github.com:GIT_USERNAME/exercises-fall2017.git
Letβs also add an upstream remote that points back to the original duckietown repo:
$ git remote add upstream git@github.com:duckietown/exercises-fall2017.git
If you type
$ git remote -v
You should now see:
origin git@github.com:GIT_USERNAME/exercises-fall2017.git (fetch)
origin git@github.com:GIT_USERNAME/exercises-fall2017.git (push)
upstream git@github.com:duckietown/exercises-fall2017.git (fetch)
upstream git@github.com:duckietown/exercises-fall2017.git (push)
Now the next time you push (without specifying a remote) you will push to your local private repo.
You should put your homework files in folder at:
DUCKIEFLEET_HOMEWORK_ROOT/homeworks/XX_homework_name/YOUR_ROBOT_NAME
Some homeworks might not require ROS, they should go in a subfolder called scripts. ROS homeworks should go in packages which are generated using the process described here: Unit K-6 - Minimal ROS node - pkg_name. For an example see DUCKIEFLEET_HOMEWORK_ROOT/homeworks/01_data_processing/shamrock.
To make your ROS packages findable by ROS you should add a symlink from your DUCKIEFLEET_HOMEWORK_ROOT to duckietown/catkin_ws/src.
When you are ready to submit your homework, you should do create a release and tag the Fall 2017 instructors/TAs group to let us know that your work is complete. This can be done through the command line or through the github web interface:
Command line:
$ git tag XX_homework_name -m"@duckietown/fall-2017-instructors-and-tas homework complete"
$ git push origin --tags
Through Github:
XX_homework_name.You may make as many releases as you like before the deadline.
Once all deadlines have passed for all institutions, we can merge all the homework. We will ask to create a βPull Requestβ from your private repo.
exercises-fall2017 repo, click the βNew pull request buttonβ.base fork: duckietown/exercises-fall2017, base: master, head fork: YOUR_GIT_NAME/exercises-fall2017, compare: YOUR_BRANCH)These instructions assume that you are ok with losing the commit history from the first homework. If not, things get a little more complicated
Fork and clone the new βhomeworkβ repository using the process above. Followed by:
$ git clone git@github.com:GIT_USERNAME/exercises-fall2017.git
Copy over your homework files from the duckiefleet-fall2017 repo into the exercises-fall2017 repo.
git rm your folder from duckiefleet-fall2017 and commit and push.
git add your folder to exercises-fall2017 and commit and push.
Clone the new duckiefleet repo
$ git clone git@github.com:duckietown/duckiefleet.git
Update the symlink you created in your duckietown repo
$ ln -sf EXERCISES_FALL2017/homeworks $DUCKIETOWN_ROOT/catkin_ws/src/name-of-the-symlink
Different than the homeworks, development for the projects will take place in the Software repo since plagiarism is not an issue here. The process is:
Create a branch from master
Develop code in that branch (note you may want to branch your branches. A good idea would be to have your own βmasterβ, e.g. βyour_project-masterβ and then do pull requests/merges into that branch as things start to work)
At the end of the project submit a pull request to master to merge your code. It may or may not get merged depending on many factors.
The following roster shows the teaching staff.
Staff: To add yourself to the roster, or to change your picture,
add a YAML file and a jpg file to the duckiefleet-fall2017 repository.
in the people/staff directory.
The sheet called βActivity Trackerβ describes specific tasks that you must do in a certain sequence. Tasks include things like βassemble your robotβ or βsign up on Githubβ.
The difference between the Areas sheet and the Task sheet is that the Task sheet contains tasks that you have to do once; instead, the Areas sheet contains ongoing activities.
In this sheet, each task is a row, and each person is a column. There is one column for each person in the class, including instructors, TAs, mentors, and students.
You have two options:
Each task in the first column is linked to the documentation that describes how to perform the task.
The colored boxes have the following meaning:
Please familiarize yourself with this spreadsheet and bookmark it in your browser.
The sheet called βAreasβ describes the points of contact for each part of this experience. These are the people that can offer support. In particular, note that we list two points of contact: one for America, and one for Europe. Moreover, there is a link to a Slack channel, which is the place where to ask for help. (Weβll get you started on Slack in just a minute.)
The organization chart (Section 8.2 - The Areas sheet) lists the primary contact for each area.
Certain documents have specific points of contacts, listed at the top. These override the listing in the organization chart.
The ways that we will support each other will depend on the type of situation. Here we will enumerate the different cases. Try to figure out which case is the most appropriate and act accordingly. These are ordered roughly in order of increasing severity.
Action: Please fix it.
The goal for the instructions is that anybody is able to follow them. Last year, we managed to have two 15-year-old students reproduce the Duckiebot from instructions.
How to edit the documentation is explained in Part B - Duckumentation documentation. In particular, the notation on how to insert a comment is explained in Section 3.3 - Notes and questions.
Note that because we use Git, we can always keep track of changes, and there is no risk of causing damage.
If you encounter typos, feel free to edit them directly.
Feel free to add additional explanations.
One thing that is very delicate is dealing with mistakes in the instructions.
A few times the following happened: there is a sequence of commands cmd1;cmd2;cmd3
and cmd2 has a mistake, and cmd2b is the right one, so that the sequence
of commands is cmd1;cmd2b;cmd3. In those situations we first just corrected
the command cmd2.
However, that created a problem: now half of the students had used cmd1;cmd2;cmd3
and half of the students had used cmd1;cmd2b;cmd3: the states had diverged.
Now chaos might arise, because there is the possibility of βforksβ.
Therefore, if a mistaken instruction is found, rather than just fixing the mistake, please add an addendum at the end of the section.
For example: βNote that instruction cmd2 is wrong; it should be cmd2b. To fix
this, please enter then command cmd4β.
Later, when everybody has gone through the instructions, the mistake is fixed and the addendum is deleted.
Action: Ask for clarification on the appropriate Slack channel. For a list of slack channels that could be helpful see Unit M-10 - Slack Channels. Once the ambiguity is clarified to your satisfaction, either you or the appropriate staff member should update the documentation if appropriate. For instructions on this see Part B - Duckumentation documentation.
Action: This is more serious than the previous.
Open an issue on the duckiefleet-fall2017 github page. Once the issue is resolved, either you or the appropriate staff member should update the documentation if appropriate. For instructions on this see Part B - Duckumentation documentation.
Action: Open an issue on the duckuments github page and provide all necessary information to reproduce it.
Action: open an issue
on the Software repository github page and provide all necessary information for reproducing the bug.
This page describes all of the helpful Slack channels and their purposes so that you can figure out where to get help.
You can also easily join the ones that you are interested in by clicking the links in this message.
| Channel | Purpose |
help-accounts |
Info about necessary accounts, such as Slack, Github, etc. |
help-assembly |
Help putting your robot together |
help-camera-calib |
Help doing the intrinsic and extrinsic calibration of your camera |
help-duckuments |
Help compiling the online documentation |
help-git |
Help with git |
help-infrastructure |
Help with software infrastructure, such as Makefiles, unit tests, continuous integration, etc. |
help-laptops |
Help getting your laptop setup with Ubuntu 16.04 |
help-parts |
Help getting the parts for the robot or replacement parts if you broke something |
help-robot-setup |
Help getting the robot setup to do basic things like be driven with a joystick |
help-ros |
Help with the Robot Operating System (ROS) |
help-wheel-calib |
Help doing your odometry calibration |
Note that we can link directly to the channels. (See list in the org sheet.) -AC
Lectures: Mon 13-15 HG F 26.5 Wed 10-12 HG E 22
Lab session: Fri 15-19 ML J 37.1
Duckielab: ML J 44.2
This was an introduction meeting.
These are the slides we showed:
Please help us making the experience better by providing feedback (can be anonymous)
Read about Duckietownβs history and watch the Duckumentary.
Start learning about Git and Github.
Montreal, Chicago? Whatβs happening?
These are the slides we presented:
a - Autonomous Vehicles: Keynote, PDF.
Please help us making the experience better by providing feedback (can be anonymous)
MyStudies is not updated; I am still on the waiting list. What should I do?
Nothing. Donβt worry, if you have received the onboarding email, you are in the class, even if you still appear in the waiting list. We will figure this out with the department.
What version of Linux do I need to install?
16.04.* (16.04.03 is the latest at time of this writing)
Do I need to install OpenCV, ROS, etc.?
Not necessary. We will provide instructions for those steps.
My laptop is not ready. Iβm having problems installing Linux on a partition.
Donβt worry, take a Duckie, and, take a breath. We have time to fix every issue. Start by asking for help in the #help-laptops channel in Slack. We will address the outstanding issues in the next classes.
How much space do I need on my Linux partition?
At least 50 GB; 200 GB are recommended for easy processing of data (logs) later in the course. If you have less space (say ~100GB), it might be wise to acquire an external hard drive to use as storage.
Are there going to be Linux training sessions?
Maybe. We didnβt plan for it, but it seems that there is a need. Subject to figuring out the logistics, we might organize an extra βlabβ session or produce a support video.
At some late hour of the night, we sent out the onboarding instructions.
Please complete the onboarding questionnaire by Tuesday, September 26, 15:00.
Today we distribute the Duckieboxes and we name the robots. In other words, we perform the Duckiebox ceremony.
If you cannot make it to this class for the Duckiebox distribution, please inform the TA, to schedule a different time.
Before arriving to class, you must think of a name for your robot.
Here are the constraints:
As members of the same community, it is important to get to know a little about each other, so to know who to rely on in times of need.
During the Duckiebox distribution ceremony, you will be asked to walk up to the board, write your name on it, and introduce yourself. Keep it very brief (30 seconds), and tell us:
You will then receive a Duckiebox from our senior staff, a simple gesture but of sempiternal glory, for which you have now become a member of the Duckietown community. This important moment will be remembered through a photograph. (If in the future you become a famous roboticist, we want to claim itβs all our merit.)
Finally, you will bring the Duckiebox to our junior staff, who will apply labels with your name and the name of the robot. They will also give you labels with your robot name for future application on your Duckiebot.
Please help us making the experience better by providing feedback (can be anonymous)
We created the channel #ethz-chitchat for questions and other random things, so that we can leave this channel #ethz-announcements only for announcements.
We sent the final list to the Department; so hopefully in a couple of days the situation on MyStudies is regularized.
The βlabβ time on Friday consists in an empty room for you to use as you wish, for example to assembe the robots together. In particular, itβs on the same floor of the Duckietown room and the IDSC lab.
The instructions for assembling the Duckiebots are here. Note that you donβt have to do the parts that we did for you: buying the parts, soldering the boards, and reproducing the image.
Expected progress: We are happy if we see everybody reaching up to Unit C-14 - RC+camera remotely by Monday October 9. You are encouraged to start very early; itβs likely that you will not receive much help on Sunday October 8β¦
A couple of announcements:
We created #ethz-chitchat for questions and other random things, so that we can leave this channel #ethz-announcements only for announcements.
MyStudies should be updated with everybodyβs names.
The βlabβ time tomorrow consists in an empty room for you to use as you wish, for example to assemble the robots together. In particular, itβs on the same floor of the Duckietown room and the IDSC lab.
The instructions for assembling the Duckiebots are here. Note that we did for you step I-2 (buying the parts) and I-3 (soldering the boards); and I-6 is optional.
We are happy if we see everybody reaching I-13 by the Monday after next. I encourage you to start sooner than later.
I see only 30 people in this channel instead of 42. Please tell your friends that now all the action is on Slack.
It looks like that the current documentation is misleading in a couple of points. This is partially due to the fact that there is some divergence between Chicago, Montreal and Zurich regarding (1) the parts given out and (2) the setup environment (which networks are available). We did the simple changes (e.g. removing the infamous part 6), but we need some more time to review the other issues. At this point, the best course of action is that you enjoy your weekend without working on Duckietown, while we spend the weekend fixing the documentation.
a - Logistics and other information: Keynote, PDF.
c - System architecture basics Keynote, PDF.
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
a - Computer Vision Basics: PDF, PowerPoint presentation.
b - Odometry Calibration: PDF, PowerPoint presentation.
Please take a video of the robot as it drives with joystick control, as described in Section 17.7 - Upload your video and upload it according to the instructions.
Example of a great video, but with a nonconforming Duckiebot.
Go forth and calibrate the camera! And get help in #help-camera-calib.
This is not ready yet! will be ready in a day or so.
Follow the instructions here to learn how to take a log.
See the list of exercises here.
Get help in #ex-data-processing.
| Robot assembly | overdue |
| Robot/laptop configuration | overdue |
| Subsection 11.13.1 - Taking a video of the joystick control | Monday Oct 16 |
| Subsection 11.13.2 - Camera calibration | Friday Oct 20 |
| Subsection 11.13.3 - Wheel calibration | not ready yet |
| Subsection 11.13.4 - Taking a log check off | Wed Oct 18 |
| Subsection 11.13.5 - Data processing exercises | Monday Oct 23 |
a - Lectures (Particle Filter) PowerPoint presentation, PDF.
b - Lectures (Lane Filter) PowerPoint presentation, PDF.
a - Lectures (Control Systems Module I) PowerPoint presentation, PDF.
b - Lectures (Control Systems Module II) PowerPoint presentation, PDF.
Points to be noted - Running what-the-duck on laptop and Duckiebot is mandatory. It helps save time in debugging errors and also is a standard way to ask for help from the staff. Keep repeating it periodically so as to keep the data up-to date - For the people lacking calibrated wheels, this serves as a reminder to calibrate the wheels and keep their duckiebot up-to date - It is advised to fill the lecture feedback form (Feedback form), so as to increase the effectiveness of the lectures - Always check the consistency of the camera calibration checkerboard before camera calibration (one has to check for the correct square size and correct distance between world and checkerboard reference)
Lecture Project Pitches PDF.
Lecture Motion Planing PDF.
A few references for planning of Andrea Censi:
Class (11:30)
Slides:
Book materials:
Acceptance emails sent
Class (10:30)
This class we are meeting in rm. 2333 Pavillion AndrΓ© Aisenstadt.
Student/Staff Introductions
Duckiebox distribution.
Go through the Duckiebox parts
Deadline: Mon Sept. 25
Class canceled. Continue working on Unit M-17 - Checkoff: Assembly and Configuration.
Class (10:30 - 11:30)
General discussion (how are things going? Sorry I was away last week.. anyone need anything?)
Intro to robotics - Modern robotic systems
The robot as a system - System architectures
Decomposing the robotics sytems into smaller pieces - autonomy architectures
Agreeing on the language that the different pieces βtalkβ - representations
Background on basic probability theory?
Slides:
Book Materials:
Lab (11:30 - 12:30)
Class (11:30 - 12:30)
Robotics middlewares - what are they and basic concepts
Introduction to the Robot Operating System (ROS)
Slides:
Book Materials:
Lab (12:30 - 1:30)
Homeworks and Checkoffs:
Class (10:30-11:30)
Book Materials (still rough drafts, will be completed soon):
Lab (11:30 - 12:30)
Liam will be in 2333
Any final help needed for Unit M-17 - Checkoff: Assembly and Configuration
Finalize Duckietown map setup
Homeworks and Checkoffs:
Unit M-17 - Checkoff: Assembly and Configuration due by 11pm.
New checkoff: βTaking a logβ will be linked later today (due Monday Oct 2)
New homework: βData processingβ will be linked later today (due Monday Oct 2)
Class (11:30 - 12:30) in Z-305
USB drive distribution
New checkoff announced: Unit M-18 - Checkoff: Take a Log
New homework announced: Unit M-20 - Homework: Data Processing (UdeM)
Letβs review the git policy for homeworks: Section 7.3 - Git policy for homeworks (TTIC/UDEM)
Announcement regarding activity spreadsheet which is now embedded here: Section 8.1 - The Activity Tracker. Feel free to just look or send Kirsten your gmail on Slack and she will give you access to it.
Slides:
Book Material:
Class (10:30 - 11:30) in Z-210
Slides:
Book Material:
Lab (11:30 - 12:30) in AA 2333
Class (11:30 - 12:30) in Z-305
Slides:
Book Material:
Lab (12:30 - 1:30) in AA 2333
Checkoff and Homework due at 11pm
Holiday no class!
Class 11:30 in Z-305
Book Material:
Lab 12:30 - 1:30 in AA2333
New Checkoff Initiated: Unit M-19 - Checkoff: Robot Calibration Deadline is Monday Oct. 23. Deliverables are: - Screenshot of your robot passing the kinematic odometry test - PR to duckiefleet repo with your 3 robot calibrations (kinematics, camera intrinsics, camera extrinsics)
Class 10:30-11:30 Z-205
Class 12:30-1:30 Z-310
Guest Lecture from Prof. James Forbes from McGill on Extended Kalman filter
slides: (pdf)
Homework Unit M-23 - Homework: Augmented Reality announced. Deadline is Friday Oct. 27 at 11pm
No class (Reading Week)
No class (Reading Week)
Class 10:30-11:30 Z-2015
Start of Introduction to SLAM
Class 11:30-12:30
End of Introduction to SLAM
Class 10:30-12:30
Project Pitches. Link to slides
Class 11:30-12:30
Checkoff Navigation initiated. Deadline Nov 15 @ 11pm.
Filtering Homework initiated. Deadline Nov 17 @ 11pm.
Project groups announced
Class 10:30-11:30
Control.
Module 1: (pptx)
Module 2: (pptx)
Module 3: (pptx)
Class 11:30-11:30
David Vazquez guest lecture
XXX
XXX
XXX
XXX
XXX
XXX
XXX
XXX
XXX
XXX
XXX
XXX
XXX
Name of TA
Things that the students should do before class.
Link to PDF and Keynote/Powerpoint materials.
Links to the units mentioned in the slides, and additional materials.
Write here the FAQs that students have following the lecture or instructions.
Name of TA
Things that the students should do before class.
Link to PDF and Keynote/Powerpoint materials.
Links to the units mentioned in the slides, and additional materials.
Write here the FAQs that students have following the lecture or instructions.
Classes take place on Mondays and Wednesdays from 9am-11am in TTIC Room 530.
The following is a list of the current checkoffs, along with the date and time when they are due. Most submissions involve uploading videos to via Dropbox File Request using the link provided in the individual checkoff.
Β
The following is a list of the current problem sets, along with the date and time when they are due.
for instructions on how the homework should be submitted.
Please keep track of how much time you spend on each problem set. We will ask you for this estimate along with other feedback at the end of each problem set.
Β
Please help us making the experience better by providing feedback (can be anonymous)
Tonight, we sent out the onboarding instructions.
Please complete the onboarding questionnaire by Thursday, September 27, 5:00pm CT
Welcome to Duckietown! This lecture constitutes the Duckiebox ceremony during which we will distribute your Duckieboxes and ask you to name your yet-to-be built Duckiebots! We will also discuss logistics related to the course, but we admit that that isnβt as exciting.
As you name your Duckiebots, please consider the following constraints:
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Note: Todayβs lecture will take place in Room 501 due to the TTIC Board Meeting
Please help us making the experience better by providing feedback (can be anonymous)
Important: Before starting these tutorials, make sure that you completed the following:
Before coming to class, please read through the following tutorials:
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
The second checkoff is due by 11:59pm CT Wednesday.
Please help us making the experience better by providing feedback (can be anonymous)
The third checkoff is due by 11:59pm CT Sunday. Note that camera calibration is necessary for the next problem set, which will be posted soon.
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Please help us making the experience better by providing feedback (can be anonymous)
Classes take place on Thursday from 1:20pm-4:20pm in NCTU Engineering Building 5 Room 635.
The following is a material of the current preview lecture, along with the date and time when they are due. Please make sure that you preview the materials every week before the class for insuring a good learning quality.
Β
Thursday October 19, 1:20pm: Week5 Material
Thursday October 26, 1:20pm: Week6 Material
The following is a list of the course material. Please preview the course every week and better if you try the lab yourself.
Β
How do you choose a good project idea? How about βwritingβ a good project idea?
This is the first lab of the semester. We are going to teach you βgitβ which is essential when becoming a professional programmer and cooperating with professional team.
What is a robotics system? Weβll introduce the concept of robotics system and some well known software architecture and middleware. Also, robotic operation system will be introduced in this class.
Seeβ Week3 Material
This week weβll continue on the topic of git. Also, a new exciting chapter has been opened. We are going to prepare our own Duckiebot! Come and join us!.
Today weβre going deeply into Duckiebotβs βmindβ. What is the algorithmn behind lane follwing? What is the secret that the duckies are so smart to drive? Here comes the answer.
Jupyter notebook is a very useful and conveniet tool while dealing with python language. We will teach you how to use it. A part of lane following algorithmn will be taught this week which is about line detector.
This week we are going to do the camera and wheel calibration. We will teach the student the theory of camera calibration, including extrinsic and intrinsic.
We will let them control the duckiebots by joystick to finish the wheel calibration. Also, we will give them chessboard for camera calibration.
What is this lecture all about?
Things that the students should do before class.
Link to PDF and Keynote/Powerpoint materials.
Please help us making the experience better by providing feedback (can be anonymous)
Links to the units mentioned in the slides, and additional materials.
FAQs that students have following the lecture or instructions.
This page describes all of the helpful Slack channels and their purposes so that you can figure out where to get help.
| Channel | Purpose |
help-accounts |
Info about necessary accounts, such as Slack, Github, etc |
help-assembly |
Help putting your robot together |
help-camera-calib |
Help doing the intrinsic and extrinsic calibration of your camera |
help-duckuments |
Help compiling the online documentation |
help-git |
Help with git |
help-infrastructure |
Help with software infrastructure, such as Makefiles, unit tests, continuous integration, etc. |
help-laptops |
Help getting your laptop setup with Ubuntu 16.04 |
help-parts |
Help getting the parts for the robot or replacement parts if you broke something |
help-robot-setup |
Help getting the robot setup to do basic things like be driven with a joystick |
help-ros |
Help with the Robot Operating System (ROS) |
help-wheel-calib |
Help doing your odometry calibration |
Note that you can link directly to the channel. (See list in the org sheet.) -AC
These are the TAs.
At ETHZ:
At TTIC:
At Montreal:
Here are the first steps for the TAs.
Note that many of these are not sequential and can be done in parallel.
Read about Duckietownβs history; watch the Duckumentary.
You have to set up:
Send an email to Kirsten Bowser (akbowser@gmail.com), with your GMail address and your Github account. She will give you further instructions.
Install Ubuntu 16.04 on your laptop, and then install ROS, Atom, Liclipse, etc.
Install the Duckuments system, so you can edit these instructions.
Andrea
Start learning about Git and Github. You donβt have to read the entirety of the following references now, but keep them βon your deskβ for later reference.
Liam?
Understand the continuous integration system.
Andrea
Build your Duckiebot according to the instructions.
Shiying (ETH)
??? (UdeM)
??? (TTIC)
As you read the instructions, keep open the Duckuments source, and note any discrepancies. You must note any unexpected thing that is not predicted from the instruction. If you donβt understand anything, please note it.
The idea is that dozens of other people will have to do the same after you, so improving the documentation is the best use of your time, and it is much more efficient than answering the same question dozens of times.
We have the following four documents outside of the duckuments:
The first job is to get your Duckiebot put together and up and running.
There is a checklist inside. You should go through the box and ensure that the parts that are supposed to be in it actually are inside.
If you are missing something, contact your local responsible and ask for help on Slack in the appropriate channel.
These sections describe the parts that are in your box.
You donβt need to buy anything - you have all the parts that you need.
Depending on how kind your instructors/TAs are, you may have to solder your boards.
You donβt need to solder anything.
You are ready to put things together now.
If you are very inexperienced with Linux/Unix/networking etc, then you may find it a valuable experience to reproduce the SD card image to βsee how the sausage is madeβ.
You probably donβt want to see how the sausage is made.
The only officially supported OS is Ubuntu 16.04. If you are not running this OS it is recommended that you make a small partition on your hard drive and install the OS.
Related parts of the book are:
Section 6.9 - How to make a partition if you want to make a partition
Now you need to clone the software repo and run things to make your robot move.
First initialize the robot:
Then get it to move!
You should record a video demonstrating that your Duckiebot is up and running. Brownie points for creative videos. Please upload your videos via the following URL:
Chicago: upload your video
Zurich: upload your video
A verified log in rosbag format uploaded to Dropbox.
Montreal deadline: Oct 4, 11pm
Zurich deadline: Oct 20, 17:00
We will log to the USB drive that you were given.
Take a 5 min log as you drive in Duckietown.
For Montreal this is rm. 2333.
For Zurich this is ML J44. Ask the TA when it is available.
For Chicago, we are still building the town, so feel free to do this at home or in the lab.
Unit C-18 - Taking and verifying a log for detailed instructions.
Section 18.5 - Verify a log for detailed instructions.
Upload the log here
Upload the log here
Upload the log here
That you have correctly cloned and followed the git procedure outline in Unit M-7 - Git usage guide for Fall 2017
That you have correctly setup your environment variables according to Section 4.1 - Environment variables (updated Sept 12)
You robot calibrations (wheels and camera (x2)) are merged to git through a PR
Slack channels: #help-wheel-calib, #help-camera-calib
Some of the services have changed and this requires a rebuild.
On both laptop and robot do:
$ cd Duckietown root
$ source environment.sh
$ make build-catkin-clean
$ make build-catkin-parallel
Remember that the git policy has changed a bit. You are probably best to re-clone the duckiefleet repo. For details see Unit M-7 - Git usage guide for Fall 2017 and particularly the section For U de M students who have already submitted homework to the previus duckiefleet-2017 repo
Donβt forget that master is now protected in duckiefleet. So make a new branch right away and call it GIT_USERNAME-devel
Follow the procedure in Unit C-16 - Wheel calibration. Once you have successfully passed the automated test, take a screen shot and post it to the slack channel #checkoffs and we will all congratulate you.
Follow the procedure in Unit C-17 - Camera calibration to do you intrinsic and extrinsic calibrations.
Take your robot to Duckietown. Put it in a lane.
On your robot execute
duckiebot $ make demo-lane-following
On your laptop do (after setting ros master to your robot):
laptop $ rqt_image_view
on your joystick you need to hit the top-right button (TODO: add picture).
On the command line you should see the output state_verbose = True
in the drop down menu select robot name/line_detector_node/image_with_lines
on the display you should see all the color-coded line detections
now open the Rviz visualizer on your laptop (after setting ros master to your robot):
laptop $ rviz
click the Add button in the bottom left.
then click the By Topic tab
then click the triangle next to /segment_list_markers underneath /duckiebot_visualizer
then double click on MarkerArray
On the display you should see the ground projected lines. Do they make sense? If not your calibration is wrong.
add a picture of what they should look like
Donβt forget at the end to submit a PR back to duckiefleet repo
Ability to perform basic operations on images
Build your first ROS package and node
Ability to process imagery live
Montreal deadline: Oct 4, 11:00pm
for instructions on how the homework should be submitted.
Complete Unit I-2 - Exercise: Basic image operations, adult version
Complete Unit I-4 - Exercise: Bag in, bag out
Complete Unit I-8 - Exercise: Live Instagram
Call your package dt-instagram-live_robot name and call your node dt-instagram-live_robot name
When you are done, take a 5min log (See Unit C-18 - Taking and verifying a log)in Duckietown (2333 in Montreal) and upload here
Ability to perform basic operations on images
Build your first ROS package and node
Ability to process imagery live
TTIC deadline: Friday, October 13 11:59pm CT
for instructions on how the homework should be submitted.
Complete Unit I-2 - Exercise: Basic image operations, adult version
Complete Unit I-3 - Exercise: Simple data analysis from a bag
Complete Unit I-4 - Exercise: Bag in, bag out
Complete Unit I-5 - Exercise: Bag thumbnails
Complete Unit I-7 - Exercise: Bag instagram
Complete Unit I-8 - Exercise: Live Instagram
Call your package dt-instagram-live_robot name and call your node dt-instagram-live_robot name.
Complete the exercise feedback form. You will receive points towards this exercise if you complete the form.
Ability to perform basic operations on images
Build your first ROS package and node
Ability to process imagery live
Slack channel to get help: #ex-data-processing
First, make sure you are in the Github Zurich team.
If your name is not here, contact Kirsten, and stop. You will not be able to do the next step.
Please clone this repository:
git@github.com:AndreaCensi/exercises-fall2017.git
using
$ git clone git@github.com:AndreaCensi/exercises-fall2017.git
This repository is writable by all Zurich people, but not readable by Chicago and Montreal. Because they grade the homework, we need to keep it secret.
We invite everybody to just push their exercises to this repository. (This is also compulsory to get help from TAs, so that the TAs can give comments that are useful for everybody.)
We have created a series of exercises that are supposed to help somebody who doesnβt know how to program in Python/Linux to get to a decent level.
We suggest the following:
First, read through the exercises and note the skills that are learned for each one.
Look at the last two: βLog Instagramβ and βLive Instagramβ. Do you think you can do them? If so, just do those two and feel free to skip the rest.
Otherwise, you have a lot to catch up. No problem. Take your time. Start with the basic exercise. TAs are here to help.
Call your package dt-instagram-live_robot name and call your node dt-instagram-live_robot name.
provide link to exercise feedback form.
Ability to project fake things from an image back into the world
Please follow the instructions on how the homework should be submitted.
Complete Unit I-9 - Exercise: Augmented Reality.
Please hold on for Github instructions.
Note that you should do a pull in Software to get all the goodies and utils described in the exercise, and exercises-fall2017 repos (in exercises-fall2017 repo this means pulling from the duckietown remote:
$ git pull upstream master
if you have followed the instructions properly.
In the exercises-fall2017 repository, you will find a template that you can use to make your own package. Basically everywhere you see littleredcorvette you would need to replace it with your robot_name.
intersection_4way.yamlThe first student to do it (from any institution) gets notoriety and a bonus.
Please follow the instructions on how the homework should be submitted.
This homework is about filtering. Either replace the existing histogram lane filter with either an Extended Kalman Filter or a Particle Filter. If you do both you will get a bonus.
Pull from master in the Software repo
Pull from the Duckietown (upstream) remote in your exercises-fall2017 repo.
We are providing a script to change all the instances of the default robot (in this case shamrock) with YOUR_ROBOT_NAME to save you time. To run navigate to the homeworks/03_filtering directory and run:
$ ./change_robot_name_everywhere.sh YOUR_ROBOT_NAME
(youβre welcomeβ¦)
In the
`homeworks/03_filtering/YOUR_ROBOT_NAME/dt_filtering_YOUR_ROBOT_NAME`
folder, the files you need to worry about are the following:
1) default.yaml: this contains the parameters that will be loaded. Hereβs what it currently looks like:
# default parameters for lane_filter/lane_filter_node
# change to your robot name below
filter:
- dt_filtering_shamrock.LaneFilterPF
- configuration:
example1: 0.2
# fill in params here
#uncomment below and comment above if you are doing EKF
# - dt_filtering_shamrock.LaneFilterEKF
# - configuration:
example2: 0.3
#fill in other params here
This parameter file tells your node to automatically load the right filter. If you are working on particle filter you can leave it the way it is and just add your parameters that you need under configuration. If you are working on EKF, comment or delete the lines for the PF and uncomment the lines for the EKF and then add your params as needed.
The other file you need to concern yourself with is in
include/dt_filtering_YOUR_ROBOT_NAME
You will need to fill in the functions that are setup for you.
As normal, tag the TAs and instructors in a release from your repo when you are ready for your work to be evaluated.
Liam?
to write
Name of Project: System Architecture
Slack channel: #devel-heroes
Software development branch: devel-sonja (Software repo) or sonja-branch (Duckuments repo)
The system architect project can be split into two missions:
Ensure that the development and integration of the projects into the system goes smoothly and that the resulting system makes sense, and is useful for future duckierations (duckie + generations).
Ensure that all teams know what their goal is and how it fits into the bigger picture
The deliverables for Mission 1 will include the following:
Mission 1 is the βwooden spoonβ level of the project.
β¦
Familiarisation with the current system status is under way.
Functional diagram has been updated to include multi-robot SLAM as alternative to single-robot SLAM to creating map.
β¦
β¦
Where there is a system, there is a want (nay, need) for optimisation. Describing a systemβs performance and resource requirements in a quantifiable way is a critical part of being able to benchmark modules and optimise the system.
Mission 2 is to formalise the description of the system characteristics, so that eventually the system performance can be optimised for some given resources.
Find a way to describe all the module requirements, specifications, etc in a formal, quantifiable language.
Find a way to calculate the requirements and specifications of a whole system or subsystem, based on the requirements and specifications of the individual modules of the system.
Find a way to calculate the optimal system configuration, based on the desired requirements and specifications of the system.
The different levels of Mission 2 are defined as follows:
Bronze standard:
Silver standard:
Gold standard:
The deliverables will then include:
β¦
Research is being done to identify some research areas that may be relevant and tools that may be helpful, in order to decide on an approach.
β¦
β¦
Make a copy of this document before editing.
Name of Project:
Team:
#XXX
Software development branch: XXX-devel
Summarize the mission for the team - What is the need that is being addressed? Do not focus on technical specifics yet.
List papers, open source code, pages in the Duckiebook, lecture slides, etc, that could be relevant in your quest.
Anything that is going to be an output. These should be quantified in terms of functionalities and performance metrics where appropriate.
Example 1: A Duckiebot detection system (functionality) with minimum precision of 0.8, a minimum recall of 0.5, a maximum latency of 50ms with maximum CPU consumption of 80% of one core (performance).
Example 2: At least 20 hours of logs Duckiebots annotated
Performance:
Silver standard:
Performance:
Gold standard:
Part of the deliverables should be:
After analysis of the resources and precise understanding of the problem you trying to solve, make a plan for how you will solve the problem. It is possible that at the start you could explore several seemingly promising avenues. However, you should converge on Bronze standard before moving to Silver standard etc.
Bronze standard:
Silver standard:
Gold standard:
A detailed description of the logs and procedure you will use to verify that the system is working the way you say it is working. In most cases this should include a regression test so that when someone changes something else, we can make sure that your thing still works as well as it used to.
Write here the current status. What works now, as opposed to what the goal is. The difference between these two is the work to be done.
it is better to have something that does not work, and a good description of what should work and why it doesnβt work, than to have something that kinda works, but nobody knows what the thing is supposed to do.
Nothing implemented.
Infinitely slow.
So and so should do such and such
The map to be used in the Fall 2017 class is shown in Figure 30.1.
The editable keynote file is in this directory of the duckuments repo. The ids on the signs correspond to the Apriltag IDs. For more details see Unit D-3 - Signage.
Welcome to the Fall 2017 projects.
Make a copy of the template 10_templates folder and paste it inside /atoms_85_fall2017_projects.
Rename the folder to the next available integer followed by the short group name. E.g.: 10_templates becomes 11_first_group_name for the first group, then 12_second_group_name for the second, and so forth.
Edit the 10-preliminary-design-document-template file name by substituting template with group-name
Open the preliminary design document and personalize the template to your group.
All groups have got their unique ID number and folders are renamed according to the following table. You are allowed and encouraged to use short names. Please merge from the master. New pull requests conflicting to this table will be rejected.
| ID | Group name | Short name |
|---|---|---|
| 11 | The Heroes | heroes |
| 12 | The Architects | smart-city |
| 13 | The Identifiers | sysid |
| 14 | The Controllers | controllers |
| 15 | The Saviors | saviors |
| 16 | The Navigators | navigators |
| 17 | Parking | parking |
| 18 | The Coordinators | explicit-coord |
| 19 | Formations and implicit coordination | implicit-coord |
| 20 | Distributed estimation | distributed-est |
| 21 | Fleet-level planning | fleet-planning |
| 22 | Transfer-learning | transfer-learning |
| 23 | Supervised learning | super-learning |
| 24 | Neural-slam | neural-slam |
| 25 | Visual-odometry | visual-odometry |
| 26 | Single-slam | single-slam |
| 27 | Anti-instagram | anti-instagram |
Instructions: What is the overarching mission of this team? You should write in one sentence.
Instructions: What is the need that is being addressed? Do not focus on technical specifics yet.
Instructions: Your rallying cry into battle. Traditionally, Duckietown uses Latin mottos. If you donβt speak Latin, please contact Jacopo Tani to have your motto translated into latin.
QUIDQUID LATINE DICTUM SIT, ALTUM VIDETUR
(Anything that is said in Latin sounds important)
Instructions: Are you going to rewrite Duckietown from scratch? Probably not. You need to decide what are the boundaries in which you want to move.
Instructions: What do you consider in scope? (e.g. having a different calibration pattern)
Instructions: What do you consider out of scope? (e.g. hardware modifications)
Instructions: What other pieces of Duckietown interact with your piece?
Instructions: List here the teams.
Time to define the particular problem that you choose to solve.
Suppose that we need to free our prince/princess from a dragon. So the mission is clear:
Mission = we must recover the prince/princess.
Now, are we going to battle the dragon, or use diplomacy?
If the first, then the problem statement becomes:
Problem statement = We need to slain a dragon.
Otherwise:
Problem statement = We need to convince the dragon to give us the prince/princess.
Suppose we choose to slain the dragon.
At this point, you might need to make some assumptions before proceeding.
All right. We are going to kill the dragon. How? Are we going to battle the dragon? Are we trying to poison him? Are we going to hire an army of mercenaries to kill the dragon for us?
The space of possible implementations / battle plans is infinite. We need to understand what will be the trade-offs.
How do you measure the functionality (what this module provides)? What are the βmetricsβ?
numbers of dragons killed per hour
Note that this is already tricky. In fact, the above is not a good metric. Maybe we kill the dragon with an explosion, and also the prince/princess is killed. A better one might be:
numbers of royals freed per hour
probability of freeing a royal per attempt
It works better if you can choose the quantities so that functionality is something that you maximize to maximize. (so that you can βmaximize performanceβ, and βminimize resourcesβ).
How do you measure the resources (what this module requires)?
numbers of knights to hire
total salary for the mercenaries.
liters of poison to buy.
average duration of the battle.
It works better if you think of these resources as something to minimize.
How would you measure the performance/resources above? If you donβt know how to measure it, it is not a good quantity to choose.
we dress up Brian as a Dragon and see how long it takes to kill him.
Can we decompose the problem?
Can you break up the solution in modules?
Note here we talk about logical modules, not the physical architecture (ROS nodes).
For each module, what is the input, and what is the output?
How is the data represented?
Note we are not talking about ROS messages vs services vs UDP vs TCP etc.
What needs to be designed?
What needs to be implemented?
What already exists and needs to be revised?
Do you need to revise the Duckietown specification?
Here, be specific about the software:is it a ROS node, a Python library, a cloud service, a batch script?
Some of the modules have been designated as infrastructure
Now, make a plan for the next phase.
What data do you need to collect?
Do you have data that needs to be annotated? What would the annotations be?
List here Duckietown packages, slides, previous projects that are relevant to your quest
List papers, open source code, software libraries, that could be relevant in your quest.
What could go wrong?
How to mitigate the risks?
Itβs time to commit on what you are building, and to make sure that it fits with everything else.
This consists of 3 parts:
Part 1: System interfaces: Does your piece fit with everything else? You will have to convince both system architect and software architect and they must sign-off on this.
Part 2: Demo and evaluation plan: Do you have a credible plan for evaluating what you are building? You will have to convince the VPs of Safety and they must sign-off on this.
Part 3: Data collection, annotation, and analysis: Do you have a credible plan for collecting, annotating and analyzing the data? You will have to convince the data czars and they must sign-off on this.
| System Architects | Sonja Brits, Andrea Censi |
| Software Architects | Breandan Considine, Liam Paull |
| Vice President of Safety | Miguel de la Iglesia, Jacopo Tani |
| Data Czars | Manfred Diaz, Jonathan Aresenault |
Please note that for this part it is necessary for the system architect and software architect to check off before you submit it. Also note that they are busy people, so itβs up to you to coordinate to make sure you get this part right and in time.
Please describe in detail what the desired functionality will be. What will happen when we click βstartβ?
Please describe for each quantity, what are reasonable target values. (The system architect will verify that these need to be coherent with others.)
Please describe any assumption you might have about the other modules, that must be verified for you to provide the functionality above.
Please describe the list of nodes that you are developing or modifying.
For each node, list the published and subscribed topics.
For each subscribed topic, describe the assumption about the latency introduced by the previous modules.
For each published topic, describe the maximum latency that you will introduce.
Please note that for this part it is necessary for the VPs for Safety to check off before you submit it. Also note that they are busy people, so itβs up to you to coordinate to make sure you get this part right and in time.
The demo is a short activity that is used to show the desired functionality, and in particular the difference between how it worked before (or not worked) and how it works now after you have done your development.
It should take a few minutes maximum for setup and running the demo.
How do you envision the demo?
What hardware components do you need?
In contrast with the demo, the formal performance evaluation can take more than a few minutes.
Ideally it should be possible to do this without human intervention, or with minimal human intervention, for both running the demo and checking the results.
Please note that for this part it is necessary for the Data Czars to check off before you submit it. Also note that they are busy people, so itβs up to you to coordinate to make sure you get this part right and in time.
How much data do you need?
How are the logs to be taken? (Manually, autonomously, etc.)
Describe any other special arrangements.
Do you need to annotate the data?
At this point, you should have you tried using thehive.ai to do it. Did you?
Are you sure they can do the annotations that you want?
Do you need to write some software to analyze the annotations?
Are you planning for it?
The βHeroesβ team is a special task force with the responsibility to make sure that βeverything worksβ and create a smooth experience for the rest of the teams, in terms of developing own projects, integration with other teams and documentation. Apart from that, each of the heroes will also have their own individual questβ¦
E PLURIBUS UNUM
(From many, unity)
The system architect is ultimately responsible:
The logical architecture
Definition of performance metrics
The system architect project can be split into two quests:
The two quests and their respective descriptions will be explained separately in this document.
With many teams working on many different parts of the system, chaos is inevitable (without divine intervention). Quest 1 is to minimise the chaos by acting as system-level wactchdog; spotting and addressing interface, contract and dependency issues between the teams.
Ensure that the development and integration of the projects into the system goes smoothly and that the resulting system makes sense, and is useful for future duckierations (duckie + generations).
Ensure that all teams know what their goal is and how it fits into the bigger picture.
The current system makes at least kind-of sense. The current system will be used as a base, onto which improvements or functionality will be added by the projects.
Challenges:
Maintaining the balance between project level scope and Duckietown level scope. For instance, teams are focused on completing their project, and might forget the greater vision of Duckietown. This might mean having to convince teams to do slightly more work, for it to be more useful to Duckietown. After all, whatβs the point of doing a project if it does not contribute to Duckietown?
Balancing priorities of quest 1 and 2. Quest 1 is crucial, and takes priority over quest 2. Therefore it will be challenging to find time (main resource) to work on quest 2.
Where there is a system, there is a want (nay, need) for optimisation. Describing a systemβs performance and resource requirements in a quantifiable way is a critical part of being able to benchmark modules and optimise the system.
The different levels of quest 2 are defined as follows:
Bronze standard:
Silver standard:
Gold standard:
Formalise the description of the system characteristics, so that eventually the system performance can be optimised for some given resources.
devel-heroes-formal-description branch on Software repositoryMake Duckietown a smarter city.
OMNES VIAE ANATUM URBEM DUCUNT
(All roads lead to Duckietown)
| Team | Reference Person |
|---|---|
| Intersection Navigation | Nicolas Lanzetti (ETHZ) |
| Parking | Samuel Nyffenegger (ETHZ) |
| Traffic controller HUB | no teams actively working on this project in Fall-2017 |
| System Architect | Sonja (ETHZ) |
| Software Architect | Breandean (UdM) |
| Knowledge | Tzarina |
We have to design a traffic lights system that integrates seamlessly and efficiently with the tiles currently used to build Duckietown. The development of a traffic lights system has to be considered as part of a bigger plan aimed to make Duckietown a smart city. A smart Duckietown has the capability of delivering wireless connectivity everywhere (Duckietown Wireless Network - DWN) in the town and power to each tile. A tile that can provide power is called a hot tile. The power grid that provides power to all the tiles is called Duckietown Power Grid (DPG). A simple use case for this infrastructure would then be the traffic lights system. Traffic lights at each intersection are powered and controlled by a Raspberry Pi with a Duckiebot-like LED Hat and 3 (or 4) LEDs. A Raspberry Pi responsible for the traffic lights at an intersection draws power from a hot tile and connects to the DWN.
To create a smarter Duckietown and provide data and power to the tiles, we will wireless networks (e.g., WiFi, Bluetooth, etc.) for data communication, and we will implement a power grid to provide power to the various devices and PDs throughout Duckietown. Since we are using common implementation of wireless networks, the rest of this design document will focus on the specifications of the power grid.
Power Grid Implementation Ideas:
Idea 1: Attach a 2-row breadboard along the edges of each tile, between the white tape and the teeth of the tile. PDs are connected to the power grid simply by inserting the two wires (+ and -) into the relative holes.
Idea 2: Attach a plastic rail to the edge of each tile, between the white tape and the teeth of the tile. The rail would carry two conductive strips (copper strips), one on each side (see image below).
Prototype: The image below shows a possible design of the plastic rail along with a compatible plug. The black part of the 3D model above constitutes the rail (sectional view) while the white part is the plug. The system is designed so that the plug, once pressed onto the rail, remains attached. The white box on the plug would contain one of the step-down converters (http://a.co/fAIAhuw) described above. This would solve the problem of having a weak 5V power grid by running 24V through the grid and stepping it down to 5V only when, and exactly where, we need it. There would be limit neither to the number of plugs nor to the position where we can attach them (even better than a breadboard in this sense). We can then design simple connectors for straight and curved tiles to make everything modular. Since the most common PD in Duckietown is a Raspberry Pi, we can design a USB plug (shown below) to make things even easier.
Enhancement 1: We can modify the plug by adding an extrusion to one side and carving its negative into the rail. This would prevent us from attaching the plug in the wrong direction, thus violating the positive/negative polarity of the conductors.


The actual voltage and amperage available at each tile/power terminal will depend on the power grid approach we choose. Regardless of the implementation, the primary functionality provided by the power grid is access to power for at each tile in the Duckietown.
The resources for this project are the parts to build the traffic lights and the power grid. Since, the specific parts and associated costs for the power grid are highly dependent on the implementation approach we decide on, we are unable to obtain specific details at this time. However, for all of the approaches, we will need enough parts to build a power grid that provides power for all of the tiles in the Duckietown.
Power Grid:
System:
Input: 12/24 V, Output: 12/24 V between tiles, 5 V on tile
Power grid and integration into the individual tiles must be designed and implemented. While the traffic lights exist, there needs to be a revised method of providing power.
May have to modify Duckietown tiles.
None, this is a hardware project.
All modules are infrastructure.
Decide connector option and wire routing.
Stability of power grid. How many traffic lights can be supported per voltage source.
None.
Specification of traffic light.
None.
What could go wrong?
How to mitigate the risks?
Estimate better models to make localization and control more efficient and robust to different configurations of the robot.
NOSCE TE IPSUM
(Know thyself)
Every Duckiebot is different in configuration.
Mission = we need to make control robust to different configuration
Problem statement = we need to identify kinematic model to make control robust enough
We make use of the no lateral slipping motion hypothesis and the pure rolling constrain as shown in the #duckiebot-modeling, to write the following equation:
\begin{align} \label{eq:mod-kin-3}\tag{1} \left\{ \begin{array}{l} \dot x_A^R &= R (\dot \varphi_R +\dot \varphi_L)/2 \\ \dot y_A^R &= 0 \\ \dot \theta &= \omega = R(\dot \varphi_R - \dot \varphi_L)/(2L) \end{array} \right., \end{align}
Further we make the assumption that for steady state that there is a linear relationship between the input voltage and the velociy of the wheel:
\begin{align} \label{eq:mod-kin-4}\tag{2} v_r=R \dot \varphi_l=c_r V_r\\ v_r=R \dot \varphi_l=c_l V_l \end{align}
This lets us rewrite equation \eqref{eq:mod-kin-3}:
\begin{align} \label{eq:mod-kin-5}\tag{3} \left\{ \begin{array}{l} \dot x_A^R &= (c_r V_r+c_l V_l)/2 \\ \dot y_A^R &= 0 \\ \dot \theta &= \omega = (c_r V_r+c_l V_l)/(2L) \end{array} \right., \end{align}
Using the assumption that we can measure $v_A$ we can determine $c_r$ by setting the voltage $V_l=0$. The same procedure can be done to get $c_l$.
Using the assumption that we can measure $\dot \theta$ we will then get the semiaxis length $L$.
Run lane follower with old version and new version with kinematic model. Drive on the track for one minute and count the number of times the bot touches the side or center line.
Metrics
We will use the performance measurement setup of the devel-control group
Parameter estimation
Mapping voltage β velocity
Duckiebots with different hardware configurations for testing
None
| Date | Task Name | Target Deliverables |
|---|---|---|
| 17/11/17 | Kick-Off | Preliminary Design Document |
| 24/11/17 | Play around | Identify current problems |
| 01/12/17 | First estimation | find paramers of robot |
| 08/12/17 | Validation | Performance measure |
| 15/12/17 | Caster wheel | Performance measure of new implementation |
| 22/12/17 | Buffer | |
| 29/12/17 | Documentation | Duckuments |
| 05/01/18 | End of Project |
What data do you need to collect?
Performances of the current implementation
the above contains a number of interesting sections of relevance to the work of this group:
exact modeling of caster wheel and the kinematic constraints it introduces (pg. 395)
different system identification procedures: parametric or nonparametric (Chapter 14); in particular, a note on Observability (pg. 337)
we want to maximize performance of control + localization. Control uses unicycle model in Frenet frame (pg. 803 of handbook of robotics)
We need to identify wheel radii (r_1, r_2: assume equal at start = r), semi-axle length L, and motors steady state parameters (mapping between voltage and angular rate, i.e. mapping between voltage and velocity once (a) wheel radius is known and no slipping hypothesis is made).
Adaptive control (pg. 147): another approach is implementing an adaptive controller. It is meant to work with plant uncertainty.
What could go wrong?
Mitigation strategy:
Make lane following more robust to model assumptions and Duckietown geometric specification violations and provide control for a different reference control.
IMPERIUM ET POTESTAS EST
(With control comes power)
Control Duckiebot on straight lane segments and curved lane segments.
Robustness to geometry (width of lane, width of lines)
Detection and stopping at red (stop) lines
Providing control for a given reference d for avoidance and intersections (but for intersections, we additionally need the pose estimation and a curvature from the navigators team)
Pose estimation and curvature on Intersections (plus navigation / coordination)
Model of Duckiebot and uncertainty quantification of parameters (System Identification)
Object avoidance involving going to the left lane
Extraction and classification of edges from images (anti-instagram)
Any hardware design
Controller for Custom maneuvers (e.g. Parking, Special intersection control)
Robustness to non existing line
System Architect
She helps us to interact with other groups. We talk with her if we change our project.
Software Architect
They give us Software guidelines to follow. They give a message definition.
Knowledge Tzarina Duckiebook
Anti-Instagram They provide classified edges (differentiation of centerline, outer lines and stop lines)
Direction of the edges (background to line vs. line to background)
Intersection Coordination (Navigators) They tell where to stop at red line. We give a message once stopped. They give pose estimation and curvature (constant) to navigate on intersection. We provide controller for straight line or standard curves.
Parking They tell where to stop at red line. We give a message once stopped
System Identification They provide model of Duckiebot.
Obstacle Avoidance Pipelines (Saviors) They provide reference d.
SLAM They might want to know some information from our pose estimation (e.g. lane width or theta).
We must keep the Duckiebots at a given distance d from center of the lane, on straight and curved roads, under bounded variations of the city geometric specifications.
Geometric specifications:
Given at every time step a reference d (the reference $\theta$ gets choosen in a way to match the reference d): \begin{equation} q_r(t)=[x_r(t),y_r(t),\theta_r(t)]^t \rightarrow [d_r(t),\theta_r(t)] \end{equation}
The following image shows the definition of the parameters $d$ and $\theta$.
($d$: distance perpendicular to the lane. 0 is defined in the center of the lane (see definition in duckumentation))
($\theta$: angle between the lane and the robot body frame. )
and given a model of the system,
define a control action:
at every time step, such that the pose (estimate of the pose) converges to the target.
Performance:
Robustness to slight changes in:
Drive on straight lane and curves without large deviations from the center of the lane.
Hardware resources:
Dependencies: see assumptions
We assume to have image space line segments extracted and classified from images.
Drive on the track for one minute and count the number of times the bot touches the side or center line. Repeat this 5 times.
Metrics Error from the reference distance d when driving straight. - Mean and variance of 5 experiments Estimate of lane width. - Estimate lane width and compare to measurement Estimate road curvature. - Estimate curvature and compare to measured radius of curve Speed - make is a control variable. Robustness to initial pose. - Run lane following using 5 different initial poses Transient error after curved section (e.g. dies in one tile length). - 5 experiments of measuring the error d when driving straight after a curved segment $\rightarrow$ Did the transient error die? Robustness to the curvature. - Run curve following on 5 lanes made of different combinations of curve tiles (left-left-right, left-right-left, β¦ ) Robustness to lane specifications - Run lane following on 5 lanes with different lane width when driving straight
Estimation of Position:
Controller:
We do not need to change the Duckietown specifications.
| Date | Task Name | Target Deliverables |
|---|---|---|
| 17/11/17 | Kick-Off | Preliminary Design Document |
| 24/11/17 | Get familiar with state of the art | Benchmark state of the art, Identifiy bottlenecks |
| 01/12/17 | Theoretical derivation of Controller and Estimator | |
| 08/12/17 | Implementation of Controller and Estimator | |
| 15/12/17 | Benchmark new implementation | Performance measure of new implementation |
| 22/12/17 | Buffer | |
| 29/12/17 | Documentation | Duckuments |
| 05/01/18 | End of Project |
Take rosbag logs every time.
Rosbag:
Curvature of road.
Vision odometry Lane detection Anti instagram
Particle Filter coded in python and useful intro to the subject
| Risk | Likelihood | Impact | Risk response Strategy | Actions required |
|---|---|---|---|---|
| Cannot estimate curvature | 2 | 5 | mitigate | Start early with testing thresholds for curvature identification |
| Cannot define distance to curve | 2 | 4 | mitigate | Try various methods to identify the distance to curve |
| Duckiebot leaves the lane after curve (current situation) | 2 | 5 | mitigate | |
| Cannot handle the inputs given by other teams | 4 | 4 | mitigate | Get more information from Sonja, Talk to other teams, Clear comments in the code for easier problem detection |
Detect obstacles, plan a route and drive around them.
URBEM ANATUM TUTIOS FACENDA
(Duckietown is to be made safer)
What is in scope
Detecting cones and duckies of different sizes (obstacles) and plan a reasonable path or stop to avoid hitting them.
Stage 1: 1 obstacle and simply stop no crossing of lines (2 cases: drive by or stop)
Stage 2: 1 obstacle, drive by without crossing of line
Stage 3: 1 obstacle, potentially cross the line
Stage 3: Multiple obstacles, crossing line if needed
What is out of scope
No obstacles in crossings
Obstacles on the middle line
Complicated situations with oncoming traffic
Stakeholders
Controllers (Lane following, adaptive curvature control)
They ensure following our desired trajectory and we can tell them to stop or reduce the speed. They provide the heading and position relative to track (for path planning)
Vehicle detection - tbd
Potentionally Anti-Instagram They provide classified edges to limit the area where we have to find obstacles.
Organizational Help - System Architect - Software Architect - Duckiebook
Reliably detect and avoid obstacles, plan a meaningful path around them or simply stop if nothing else is possible.
Robustness to changes in:
Stage 0: Collect enough data and annotate them
Stage 1: Develop a first obstacle detection algorithm
Stage 2: Agree on final internal and external interface
Stage 3: from now on, obstacle detection and trajectory planning can be developed in parallel
Stage 4: handle the case(s) involving: 1 obstacle, no crossing of lines (2 cases: drive by or stop) β simple logical conditions
Stage 5: handle the case(s) involving: 1 obstacle, crossing line if needed (1 case: should always be possible to drive by) β Either with grid map or obstacle coordinates
Stage 6: handle the case(s) involving: Multiple obstacles, crossing line if needed
Stage 7: verify the whole system
Detection 2D space
(Curvature of upcoming track)
Output:
Reconstruction of 3D obstacle coordinates and radius
Extrinsics
Output:
Avoid obstacle
Lane information
Output:
No need to revise duckietown specifications
| Date | Task Name | Target Deliverables |
|---|---|---|
| 15/11/17 | First Meeting | Preliminary Design Document |
| 17/11/17 | Record first bags | Pictures and Raw bag data |
| 22/11/17 | Exchange of ideas | Basic concept |
| 29/11/17 | Knowing Interfaces and State of the Art | Fine concept |
| 06/12/17 | First implementation | |
| β¦ | Testing | Optimized Code |
| β¦ | Documentation | Duckuments |
| 21/12/17 | End of Project |
Images of duckies on the road.
Video of a duckiebot in duckietown with recordings of the different stages.
To log:
Label obstacles
Image processing
feature extraction
MIT2016 object detection
Lane detection
Anti instagram
OpenCV (filtering, color and edge detection)
Interfaces (control approach of trajectory)
Computation power
| Risk | Likelihood (1-10) | Impact | Actions required |
|---|---|---|---|
| Non robust State Estimation | tbd | very high | Communication/Collaboration with controlling subteam |
| Failure of following our desired control commands | tbd | very high | Communication/Collaboration with controlling subteam |
| Lack in computation power | 5 | high | early testing of whole system on duckiebot |
| Failure in duckie detection | 4 | extremely high | thorough testing on bags |
| Erroneously detecting the middle lane as duckie | 7 | middle | more sophisticated detection algorithm |
Implement parking feature and design specifications (feature and physical)
Quidquid latine dictum sit, altum videtur (anything that is said in Latin sounds important)
Change template motto to group motto.
Implement parking feature and design specifications (feature and physical)
We need to park N Duckiebots in a designated area in which they are able enter and exit in an efficient manner.
Yes, we need to add parking lot specifications.
A collection of ROS nodes.
Yes, we will include infrastructure modules to specify parking lot specifications.
No
Coordinate intersection navigation safely through explicit communication.
IN HOC SIGNO VINCES
(with this sign, you will win)
Employ LEDs based communication to efficiently manage traffic at intersections. First, LEDs should be used to reliably communicate Duckiebots positions and/or intentions. Then, optimal control theory or game theory might be considered to safely clear the intersection in reasonable time. By safely, we mean that no collisions occur (100% success). This requires robustness in message interpretation.
We aim to solve the problem of clearing the intersection in max. 60 seconds (meaning that even in the case of four Duckiebots participating we can solve the problem below this time). A decentralized solution should be implemented.
The following aims are identified:
The following aims are beyond the scope of the project:
The following pieces of Duckietown will be interacting with the project:
Duckiebots must be able to cross an intersection in the defined reasonable time and without collisions.
The following problems are to be tackled:
The following assumptions are made for the LEDs communication:
The following assumptions are made for the coordination:
Firstly, the existing code and see its limits should be tested. Benchmark tests have to be designed in order to carefully measure the performances of the implementation. Then, the literature should check to see what has already been implemented.
The problem can be splitted in LED-based communication and intersection coordination. We further distinguish between intersections with traffic lights and intersections without traffic lights (recall that it is assumed that the Duckiebots know the intersections they entering). In all cases we aim for a decentralized solution.
Possible approaches for LEDs communication:
Possible approaches for intersection coordination without traffic lights:
Possible approach for intersection coordination with traffic lights:
Last yearβs approach: Communication works through LEDs blinking at different frequencies. Colors are just for human understanding and are operational meaningless. See last yearβs documentation for further details.
Alternative approach 0: Communication works through LEDs blinking at different frequencies, colors, and patterns.
Last yearβs approach: See last yearβs documentation.
Alternative approach 1:
Assumption: every Duckiebot does not see on the left. Here, Duckiebots do not need to turn in place to see the Duckiebot on the left
There are two messages:
The coordination plan works as follows:
If Duckiebot:
If 4 Duckiebots are at intersection, that means, every Duckiebot is emitting signals A and B, each Duckiebot turns off with a certain probability (say 25%), so that one is going to see nobody on the right and will navigate the intersection.
If one Duckiebot does not see any kind of signal (A or B) it will proceed to navigate the intersection.
One case still need to be analyzed, i.e., the one in which we have 2 Duckiebots one opposite to the other. In this case:
Then each Duckiebot turns off with some probability (say 50%) so that it will see no more signals and then navigate the intersection, as explained in case 3.
Alternative approach 2: There are four messages:
In the following, let you be a Duckiebot.
You currently have no color. You stop at the intersection, turn left 20Β° in order to see all existing bots. You have no lights on at the moment. Possible scenarios:
(after the above algorithm, we could also implement the below to increase throughout)
Back lights: If you are entering the intersection, turn back lights to green. Else (negotiating, or waiting), turn off back lights.
Duckiebots behind a Duckiebot that stopped: If see a green back light, get ready to run, turn front lights to yellow and immediately enter the intersection after the car in front of you. If no green light on the back of the Duckiebot in front of you, stop at the red line as usual and start executing the intersection algorithm.
Improvements: We could possibly give frequency to back lights to let more than 2 Duckiebots enter the intersection, such that the first Duckiebot that is about to enter the intersection has constant green lights on the back, 2nd one behind it has green back lights with frequency x, and 3rd behind 2nd one knows it is the 3rd since it sees green light with frequency so it also enters the intersection immediately (and has lights off on the back so that 4th has to stop at the red line and execute the default intersection algorithm).
Therefore, this alternative needs 3 different signals, such as green, red, yellow and it will also use the no color case as a 4th signal.
Alternative approach 3:
This approach consists of is an exponential backoff model (assuming that two other visible cars is a low enough number to have it run quickly enough). Here:
The intersection policy works as follows:
START:
CHECKING:
WAIT:
Metrics for LEDs communication:
Metrics for coordination:
The following resources are needed in order to test the behaviour:
Performance measurements for LEDs communication:
We put one Duckiebot in a lane at the intersection, one Duckiebot on the opposite lane across the intersection, and then one on the right. First the opposite and then the right Duckiebot will emit signals, the observer will receive them. We will see what happens if we let LED-detector-node run. We should be able to detect in which regions are LEDs blinking (See f23-presentation, Minute 11:20 in Google Drive).
We put four Duckiebots at an intersection (with and without traffic light) and let them communicate. This allows to test whether the communication was successful or failed and the time needed to perform it.
Performance measurements for coordination:
Following subprojects are detected:
LED communication
Coordination at intersection
The Duckietown specification do not need to be revisited.
The software will be organized as follows:
The existing code has already a similar structure, meaning that part of the code might be reused.
No infrastructure modules are needed.
We are going to implement and test the alternative coordination algorithms along with the last yearβs algorithm and check safety and performance metrics.
| Week | Task | Expected Outcome |
| 12.11.17- 19.11.17 | Kick-off Hardware up to date Specifications for other groups First tests | Preliminary design |
| 20.11.17- 26.11.17 | Test existing code Implementing new algorithms | Benchmark results First implementation |
| 27.11.17- 03.12.17 | Implementing new algorithms continues Benchmark the new algorithms | Performance of the new algorithmsDeadline for Chicago |
| 04.12.17- 10.12.17 | Try more sophisticated implementations such as algorithms that include communication with the bots behind, through back lights | Implementation |
| 11.12.17- 17.12.17 | Benchmark new algorithms | Benchmark results |
| 18.12.17- 24.12.17 | TBD | TBD |
| 25.12.17- 31.12.17 | TBD | TBD |
| 01.01.18- 07.01.18 | TBD | End of the project |
No data needs to be annotated.
These are some of the important features used in the past course:
Literature and codes from the class of 2016 at MIT. Additionally, research papers in computer vision (How to detect LEDs? How to use the camera? Etc.) and coordination (how to optimally coordinate a fleet? Etc.) might be of interest.
The following risks for LEDs communication are identified:
Detecting colors. A possible solution would be to test colors and frequency to see which one is more robust.
Limited visibility. Slightly turning in place (and then coming back to the initial position) might alleviate this problem.
The following risks for coordination are identified:
We can summarize the risks in the following table.
| Risk | Potential error | Consequence | Likelihood | Impact | Risk Priority Number (0-100) | Actions required |
| Detecting color | Camera does not recognize colors | Signal cannot be encoded in color | 7 | 9 | 63 | Alternative interpreter strategy. |
| Visibility | Limited visibility from DB camera | Limited exchange of messages | 10 | 8 | 80 | Strategy that does not need information from every DB |
| Synchronization before messages | Problems when bots arrive at the intersection at different times | Wrong communication | 3 | 10 | 30 | Take into account while formulate strategy. Safety procedure needed. |
| Synchronization during messages | Problems when bots detect signals from other bots while bots are changing signals. Problems if bot detects signals while other bots are changing signals | Wrong communication | 3 | 10 | 30 | Safety procedure for this case. |
Formation keeping and collision avoidance using implicit communication
ALIIS VIVERE
(Live for others)
Mission: Formation keeping and intersection navigation with Duckiebots just using implicit communication.
Problem statement: Detect other Duckiebots, follow the leader in a secure distance through Duckietown and coordinate intersections using implicit communication.
Formation:
Implicit Coordination
Data Collection
Detection -Instance-level 2D/3D Bounding box + pose estimation.
Formation keeping and intersection coordination
Robustness of Formation keeping and Vehicle Detection
Implicit coordination
Detection of Duckiebot
Estimation of relative pose
Controller
Intersection behavior
(Special T-intersection with only one stop in Duckietown)
None
| Date | Task Name | Target Deliverables |
| 17/11/2017 | Kick-Off | Preliminary Design Document |
| 24/11/2017 | Review state of the art, theoretical formalisation | List of what has to be implemented and how and what can be reused from last year. Benchmarking current state |
| 1/12/2017 | Implementation | - |
| 8/12/2017 | Implementation and Testing | Code |
| 15/12/2017 | Evaluation, Optimization | Performance measurement |
| 22/12/2017 | Buffer | - |
| 29/12/2017 | Duckumentation | Duckuments |
| 05/01/2018 | End of project | - |
End of project
Annotated images
Duckiebots in bounded boxes under different conditions (Illumination, LEDs on/off, Duckiebot configurations, β¦)
Exisiting intersection safty rules (Duckumentation)
Existing vehicle detection algorithm (Code, Duckumentation)
Robotics Handbook: Ideas for controller for βfollow-the-leaderβ or driving in traffic on pg. 808
Likelihood from 1-10, where 10 is very likely and 1 very unlikely.
Impact from 1-10, where 10 is a huge negative impact and 1 not so bad.
| Event | Likelihood | Impact | Risk response Strategy (avoid, transfer, mitigate, acceptance) | Actions required |
| False Negative in Vehicle detection can lead to crash | 4 | 9 | mitigate | The effect of FPs are less bad than those of FNs. Punish FNs more in an eventual cost function. |
| False Positive in Vehicle detection makes bot stand still. | 4 | 2 | accept | - |
| Inaccurate estimation of distance between bots | 3 | 7 | mitigate | Early testing, big enough safety margin, improve controller |
| Collision of 2 duckiebots in an intersection | 3+4 | 9 | mitigate | Combination of Event 1 and 3 |
Enable Duckiebots to communicate with each other wirelessly
UBI UNUM IBI OMNES
(Where there is one, there is everybody)
Communication in a centralized network: Communication between Duckiebots in one Duckietown Define interfaces for communication Performance testing on the communication system One network for one Duckietown * TBD: does anything need to be synchronized?
Option 1: Communication in a centralized network with a redundant centralized component (multiple routers)
Option 2: Communication in a de-centralized network (ad-hoc)
Mission = Enable Duckiebots to communicate wirelessly
Problem Statement = Create a communication framework for the Duckiebots
Step 0: Define contracts with distributed estimation / fleet planning teams
Step 1: Create the communication framework for the duckiebots and test it on a centralized network
Step 2 (Optional): Test the communication framework using a redundant centralized network
Step 2 (Optional): Test the communication framework using a de-centralized network
Bandwidth definition:
Libraries: Serialization Messaging
Integration: Libraries integration into component Communication framework into Duckietown (repository, duckumentation, etc. β contact Integration Heroes) * Creation of the WiFi network (centralized vs. decentralized)
Testing: * Measuring and visualization of performance metrics
Fleet planning Send data (SLAM, etc.) Distribute data through the network
Distributed Estimation Receive data (local maps built from each Duckiebots) Send data (local maps and/or global map)
Integration Heroes Duckumentation Contract negotiation * External libraries (Protocol buffers, ZMQ, etc.)
Network configuration (centralized/decentralized)
Testing
Tasks:
Deliverables:
Tasks:
Deliverables:
Tasks:
Deliverables:
Tasks:
Deliverables:
Tasks:
Deliverables:
Tasks:
Deliverables:
Tasks:
Deliverables:
Tasks:
Deliverables:
TBD β> other groups
No data to be annotated.
WiFi specs (duckumentation)
Suggestions on Slack channel: 1. MAVLink (born for UAVs also used for other robots) 2. ROMANO (not so goodβ¦) 3. DDS (standard for ROS 2.0)
Computer Networks:
Serialization and Messaging:
Google python coding style guide:
Redundant routers (rollover, cascading):
Static code analysis in python:
Pylint configuration:
Possible Risks? Network problems (ad-hoc: unstable network, low bandwidth, high latency, β¦) No ad-hoc network solution Sizable amount of redundant data sent over wifi (chaos) Synchronization
How to mitigate the risks? Synchronization not part of networking β contract Contracts to prevent redundancy
βCreate the mobility-on-demand service for Duckietown.β
VICTORIA CONCORDIA CRESCIT
(Victory through harmony)
We need to combine parts of many different stakeholders to achieve planning for a fleet of duckiebots.
Necessary steps:
Functionality includes:
Metrics:
See Figure 13.1.
Duckiefleet - request handling server:
Customer - request handling server:
Request handling server - Duckiefleet:
No revision of existing duckietown specification necessary.
Revisit visualization of Duckiebots on map and adapt it for visualization of N Duckiebots
None
| Week of | Task | Deliverable |
| 13.11.2017 | Project kick-off and planning | Preliminary Design Document |
| 20.11.2017 | Look at state of current infrastructure | Running visualization of 1 duckiebot on map as currently implemented |
| 27.11.2017 | Visualization of n duckiebots | |
| 04.12.2017 | Mission planner, implement m-stochastic queue median policy (or similar, tbd with Claudio) | |
| 11.12.2017 | ...continued | Run test cases (e.g. send n reference locations to n duckiebots) |
| 18.12.2017 | ...continued | Run test cases (e.g. send n reference locations to n duckiebots) |
| 25.12.2017 | Implement customer request handling | Run test cases to establish reliable customer request handling routine |
| 01.01.2018 | Physical visualization of status, ETH formation | Verify that it works |
None
None
According to meeting notes:
Papers:
βFundamental performance limits and efficient polices for Transportation-On-Demand systemsβ by Marco Pavone, Kyle Treleaven and Emilio Frazzoli link
We will use unsupervised learning to build a depth estimation system for Duckietown. The application could be building a point cloud map for Duckietown for mappingβour overall plan is to have a serviceable deep network that is trained end-to-end with no ground truth data. We will also be using the Movidius chip, hopefully learning to how integrate it well into the current system for future users.
CARPE DIEM
(Seize the day)
The scope of the project is to use recent work in unsupervised depth estimation as well as training data we gather in Duckietown to create a fully unsupervised depth estimation system.
The deep network and use of Movidius chip.
Localization. We plan on utilizing the april tags for localization. Also hardware modificationβwe plan to do fully monocular depth.
All of the SLAM teams could (potentially) benefit from our system, though it might be non-trivial to integrate it.
We need to train a deep neural network to predict depth from monocular video (with no GT depth!)
We assume that itβs possible to collect diverse enough training data in Duckietown to train a deep CNN (outside data like KITTI might not transfer to this task).
We will start with the βUnsupervised Learning of Depth and Ego-Motion from Videoβ paper and modify the framework for duckietown.
Thereβs often a tradeoff beteen size of network and performanceβthe Movidius chip is more limited than our usual NVidia GPUs.
We will need to develop some confidence measures for our depth map. Time permitting, we would like to build a point cloud map of Duckietown.
We will need the Movidius chip for best performanceβour metrics are the size of the network (including activations) and making sure that inference is real-time.
Measuring performance here is tricky since there is no way to obtain ground truth depth data from Duckietown (aside from using a range sensor not available for duckiebots). We will likely develop a surrogate metric based on april tags.
Since itβs an end-to-end neural network itβs all in one logical module, but we could split up the design of the architecture from the way we use the data (e.g. data augmentation)
Input: RGB Image Output: Depth map (potentially relative depth, discretized)
The architecture and the data collecation and augmentation schemes need to be designed. Tensorflow implementation of architecture is what needs to be implemented. There already exists open source code for unsupervised learning of depth paper (linked below).
Do you need to revise the Duckietown specification? N/A
The software will be a simple end-to-end CNN going from frames to a depth map (likely a node publishing a depth map)
Next phase is to start collecting data and experimenting with architectures in Tensorflow.
Video of duckiebot traversing duckietown.
No data annotation necessary.
All of the camera geometry and computer vision notes.
Itβs possible that training a deep neural network on only duckietown data will be difficult. We will also consider using the KITTI dataset as additional training data.
Make the Duckiebot robust to illumination variation.
Line detection includes different colors of lines, we need to be able to detect these colors accurately.
SEMPER VIGILANS
(Always vigilant)
| System architect | She helps us to interact with other groups. We talk with her if we change our project. |
| The controllers | A.A. is the interaction person to confirm that we fulfil their requests regarding: frequency, latency, accuracy (resolution), maximum false positives, maximum misclassification error (confusion matrix). |
| The Parking | We will determine together the best color for the parking lot lines if needed. |
| The Navigators | At the moment the Duckiebot is controlled open loop at intersections. This should be improved. Probably they need line detection in a certain way. We can help and figure together out what procedure would be the best |
One of the central problems in autonomous mode of the duckiebot is the line detection. Line detection though is very sensitive to illumination variation. This problem has been solved by a color transformation called βAnti-Instagramβ. The current illumination correction algorithm, however, is not working well enough. This affects the extraction of the line segments since the extract-line-segments algorithm is very sensitive to illumination changes (e.g. shadow, bright light, specular reflections). There are several reasons why the current implementation fails:
Lighting:
Duckietown Condition:
Training data:
We also assume the current method of extracting lane pose from segments is accurate.
Understand current system
Use geometric information to better determine the actual existing colors.
Use time information/parameters from earlier illumination corrections to improve robustness (Online learning).
Further improvements:
We are assuming to have ground truth pictures. Then it is possible by processing the same picture with our algorithm and compare it to the ground truth to calculate an error.
We are going to consider true positives, true negatives, false positives and false negatives. This can be done either for only one color/one feature (dashed lines, continuous linesβ¦) or for the whole process at once which means everything should be classified properly. In addition, we would like to measure the accuracy of the lane pose estimation for our algorithm vs. a ground truth, this can be a Euclidean distance.
Costs:
Resources:
Dependencies:
As stated above we are only involved in determining the best fitting color of the (not yet installed) parking lot lines. All the other colors and the environment are assumed to be given.
NO
| Date [MM/DD/YYYY] | Task | Who | Target Deliverables |
|---|---|---|---|
| 11/20/2017 | Finish the Preliminary Document, Peer Reading of other team members | Milan, Christoph | Preliminary Document |
| 11/27/2017 | Investigate why current algorithm fails. | Milan, Christoph | Create detailed description when the algorithm fails and when it works. Make it understandable why for everyone |
| 11/27/2017 | Create data annotation, check how website works | David, Shengjie | Get 1000 annotated pictures or have a specific date when these images are delivered. |
| 12/1/2017 | Find out what colorspace is the best for the current algorithm | best color space, performance analysis | |
| 12/4/2017 | Find out whatβs the best clustering method based on best color space, is the best color space still the best? | Best clustering method, best color space, performance analysis | |
| 12/4/2017 | Include geometry in the current color transformation algorithm | Performance analysis | |
| 1/8/2018 | Implement an online system | Performance analysis of supervised system |
Around 1000 pictures with the Duckiebot camera from a Duckiebot perspective in Duckietown. The pictures have to be from different environment conditions (illumination, specular light)
The data collected above has to be annotated. The annotations should state what type and what color it is.
The whole sum of nodes within the β10-lane-controlβ folder will be within the scope of this project. This is:
anti_instagramground_projectionlane_controllane_filterline_detectorcomplete_image_pipelineThe goal is to test transfer learning algorithms trained on simulator and test on the real duckietown.
Solving control problems in reality is hard due to sparse reward signals, expensive data acquisition and the danger of breaking the robot during exploration. It is comparatively easier to train the policy in a simulator as we can βspeed upβ the reality, and there are no inherent dangers of running arbitrary policies. But policies trained on simulator do not necessarily transfer directly onto the real world, and our goal is try and bridge this gap
IPSA SCIENTIA POTESTAS EST,
UT TRANSIRE CALLIDUS EST
(Knowledge is power, transferring that is clever)
Final goal: Implement DARLA or https://arxiv.org/pdf/1703.06907.pdf or https://arxiv.org/pdf/1710.06537.pdf
Testing out different RL algorithms, (or unsupervised feature learning algorithms, if any)
Methods for transferring from simulator to duckietown
Baselining (?)
Hardware modifications
Design modifications that might be needed
Anything not involving the control of the duckiebot
Simulator (Maxime and Florian)
Supervised Learning (Rithesh and Alex)
Running NN models on the bot (Neural stick ?) who is in charge?
File with all projects click
We will replace the part of the pipeline that uses raw images and give motor controls by using a model trained in a simulator. The model policy will be able to do lane following and navigation. The model can hopefully generalize to imperfect camera calibration, motor calibration and different light conditions.
Mission: Efficient transfer of algos trained on simulators
Problem statement:
We need the simulation to be close enough to real world so as to be transferable.
We need to run inference on the robot at test time.
Maybe allow for fine-tuning in the real world (on a small dataset)
Lane following or route following (i.e. follow a given route through duckietown and obey traffic laws)
Metrics:
Reward function description (probable):
Module 1: Policy mapping raw images βΆ actions (or Module 1a: policy βΆ disentangled representation βΆ actions)
Module 2: Actions βΆ motor voltages (Joy Mapper node)
Module 3: Training module (Also introduces domain/dynamics randomization in the simulator)
Module 1: subscribe to raw camera images and publish actions (forward/backward/turn left/turn right = float values)
Module 2: already provided in duckietown stack
Module 3:
See Modules and Interfaces above.
During training, the modules will be written in Python.
At test time, modules 1,2 will be deployed as a ROS node during test.
Simulator
The simulator will annotate data automatically by providing ground truth information about the duckiebot and the environment
To be confirmed⦠We probably need to implement a module that does that.
If needed, then semantically segmented images would useful
Simulator
PyTorch rl codebase, OpenAI Gym, OpenAI Baselines
DARLA: https://arxiv.org/abs/1707.08475
Transfer by randomization/generalization:
UNREAL: https://arxiv.org/abs/1611.05397
Socially Aware Motion planning: https://arxiv.org/abs/1703.08862
Deploying the policy trained in simulator will break the bots in real duckietown.
Risk mitigation:
to write
duckieteamcreate-machinesThis program creates the machines file, using the data contained in the scuderia database.
Run as follows:
$ rosrun duckieteam create-machines
create-rosterthis program is unfinished
duckietownduckietown_msgseasy_algoThe problem that this package solves is the need to easily refer to the configuration of algorithms, and objects in general.
Hereβs one case: a node requires a line detector; a line detector has a dozen parameters. Do we want to put a dozen parameters in the node?
Or, think about the case where there are different line detector classes. Now, there must be a parameter in the node that tells which class to instantiate.
All of this business can be simplified by the features of EasyAlgo.
The approach is to declare that there is a βfamilyβ of things, and to provide
the description of what those things are in YAML files.
These files can be anywhere in catkin_ws.
To declare the family of line detectors we create a file
line_detector.easy_algo_family.yaml,
with the following content:
interface: line_detector.LineDetectorInterface
description: These are the line detectors.
This tells the system that there exists a family called line_detector (from the filename)
and that the objects in the family are instances of line_detector.LineDetectorInterface.
Then, we can write the configuration files that describe the particular instances.
These files have the pattern instance name.family name.yaml.
Continuing the example, suppose that there is a line_detector.ConcreteLineDetector with two parameters, alpha and beta.
Then we can define a baseline object by creating a file called baseline.line_detector.yaml,
containing the following:
description: The baseline detector
constructor: line_detector.ConcreteLineDetector
parameters:
alpha: 1.0
beta: 1.0
Similarly, we can define an experiment object in a file called experiment.line_detector.yaml:
description: My experiment with wild parameters
constructor: line_detector.ConcreteLineDetector
parameters:
alpha: 2.0
beta: 13.0
These configuration files are parsed by EasyAlgo.
The user can see the list of family by using:
$ rosrun easy_algo summary
To see the list of instances available for a family, use:
$ rosrun easy_algo summary family name
From the developerβs point of view, it is possibly
to access the code using the EasyAlgoDB class.
Create an instance using get_easy_algo_db():
from easy_algo import get_easy_algo_db
algo_db = get_easy_algo_db()
Then create an instance like the following:
line_detector_instance = algo_db.create_instance('line_detector', 'baseline')
It is also possible to query the database using the function query:
# iterate over all possible line detectors
available = algo_db.query('line_detector', '*'
for name in available:
line_detector = algo_db.create_instance('line_detector', name)
easy_logsThe package includes a few programs to query the database: find, summary, and details.
They all take the same form:
$ rosrun easy_logs program query
where query is a query expressed in the selector language (Section 5.3 - Selector language).
Use the summary program to display a summary of the logs available:
$ rosrun easy_logs summary "*dp3tele*"
| # Log name date length vehicle name valid
| -- ------------------------------ ---------- ------ ------------ -----------
| 0 20160504-dp3tele1-thing 2016-05-04 294 s thing Yes.
| 1 20160506-dp3tele1-nikola 2016-05-06 321 s nikola Yes.
| 2 2016-04-29-dp3tele-neptunus-0 2016-04-29 119 s neptunus Yes.
| 3 20160503-dp3tele1-pipquack 2016-05-03 352 s pipquack Yes.
| 4 20160503-dp3tele1-redrover 2016-05-03 384 s redrover Yes.
| 5 20160504-dp3tele1-penguin None (none) None Not indexed
| 6 20160430-dp3tele-3-quackmobile 2016-04-30 119 s quackmobile Yes.
The find command is similar to summary, but it only displays the filenames:
$ rosrun easy_logs find vehicle:oreo
.../logs/20160400-phase3-dp3-demos/dp3auto_2016-04-29-19-58-57_2-oreo.bag
.../logs/20160400-phase3-dp3-demos/dp3auto_2016-04-29-19-56-57_1-oreo.bag
.../logs/20160400-phase3-dp3-demos/dp3tele_2016-04-29-19-29-57_1-oreo.bag
.../logs/20160210-M02_DPRC/onasafari/201602DD-onasafari-oreo-RCDP5-log_out.bag
.../logs/20160400-phase3-dp3-demos/dp3tele_2016-04-29-19-27-57_0-oreo.bag
.../pictures/M03_04-demo_or_die/onasafari/oreo_line_follow.bag
.../logs/20160400-phase3-dp3-demos/dp3auto_2016-04-29-19-54-57_0-oreo.bag
.../logs/20160400-phase3-dp3-demos/dp3tele_2016-04-29-19-31-57_2-oreo.bag
The details command shows a detailed view of the data structure:
$ rosrun easy_logs details dp3auto_2016-04-29-19-58-57_2-oreo
- - [log_name, dp3auto_2016-04-29-19-58-57_2-oreo]
- [filename, ...]
- [map_name, null]
- [description, null]
- [vehicle, oreo]
- [date, '2016-04-29']
- [length, 112.987947]
- [size, 642088366]
- - bag_info
- compression: none
duration: 112.987947
end: 1461960050.639
indexed: true
messages: 15443
size: 642088366
start: 1461959937.651054
topics:
- {messages: 107, topic: /diagnostics,
type: diagnostic_msgs/DiagnosticArray}
- {messages: 8, topic: /oreo/LED_detector_node/switch,
type: duckietown_msgs/BoolStamped}
- {messages: 9, topic: /oreo/apriltags_global_node/apriltags,
type: duckietown_msgs/AprilTags}
- {messages: 9, topic: /oreo/apriltags_global_node/tags_image,
type: sensor_msgs/Image}
...
Here are some examples for the query language (Table 5.1).
Show all the Ferrari logs:
$ rosrun easy_logs summary vehicle:ferrari
All the logs of length less than 45 s:
$ rosrun easy_logs summary "length:<45"
All the invalid logs:
$ rosrun easy_logs summary "length:<45,valid:False"
All the invalid logs of length less than 45 s:
$ rosrun easy_logs summary "length:<45,valid:False"
| expression | example | explanation |
attribute:expr |
vehicle:ferrari |
Checks that the attribute [attribute of the object
satisfies the expression in expr |
>lower bound |
>10 |
Lower bound |
<upper bound |
<1 |
Upper bound |
expr1,expr2 |
>10,<20 |
And between two expressions |
expr1+expr2 |
<5+>10 |
Or between two expressions |
pattern |
*ferrari* |
Other strings are interpreted as wildcard patterns. |
downloadThe command-line program download downloads requires
logs from the cloud.
The syntax is:
$ rosrun easy_logs download log name
For example:
$ rosrun easy_logs download 20160223-amadoa-amadobot-RCDP2
If the file 20160223-amadoa-amadobot-RCDP2.bag is not available
locally, it is downloaded.
The database of URLs is at the file dropbox.urls.yaml in the package easy_node.
A typical use case would be the following, in which a script needs a log with which to work.
Using the download program we declare that we need
the log. Then, we use find to find the path.
#!/bin/bash
set -ex
# We need the log to proceed
rosrun easy_logs download 20160223-amadoa-amadobot-RCDP2
# Here, we know that we have the log. We use `find` to get the filename.
filename=`rosrun easy_logs find 20160223-amadoa-amadobot-RCDP2`
# We can now use the log
vdir ${filename}
How do you know which logs are in the cloud?
Run the following to download a database of all the logs available in the cloud:
$ make cloud-download
You can query the DB by using summary with the option --cloud:
$ rosrun easy_logs summary --cloud '*RCDP6*'
| # Log name date length vehicle name
| -- --------------------------------------- ---------- ------ ------------
| 0 20160122-censi-ferrari-RCDP6-catliu 2016-02-28 194 s ferrari
| 1 20160122-censi-ferrari-RCDP6-joe-wl 2016-02-27 196 s ferrari
| 2 20160228-sanguk-setlist-RCDP6-sangukbo 2016-03-02 193 s ferrari
| 3 20160122-censi-ferrari-RCDP6-eharbitz 2016-02-27 198 s ferrari
| 4 20160122-censi-ferrari-RCDP6-teddy 2016-02-28 193 s ferrari
| 5 20160122-censi-ferrari-RCDP6-jenshen 2016-02-29 83 s ferrari
Then, you can download locally using:
$ rosrun easy_logs download 20160122-censi-ferrari-RCDP6-teddy
Once it is downloaded, the log becomes part of the local database.
expr/shuffle shuffles the order of the logs in expr.
Give me all the oreo logs, in random order:
$ rosrun easy_logs summary vehicle:oreo/shuffle
expr/[i] takes the i-th entry.
Give me the first log:
$ rosrun easy_logs summary "vehicle:oreo/[0]"
Give me a random log; i.e. the first of a random list.
$ rosrun easy_logs summary "vehicle:oreo/shuffle/[0]"
You can use the exact Python syntax for indexing, including [a:], [:b], [a:b], [a:b:c], etc.
Give me three random logs:
$ rosrun easy_logs summary "all/shuffle/[:3]"
To implement.
You can ask for only a part of a log using the syntax:
expr/{start:stop}
expr/{start:}
expr/{:stop}
where start and stop are in time relative to the start of the log.
For example, βgive me all the first 1-second intervals of the logsβ is
all/{:1}
Cut the first 3 seconds of all the logs:
all/{3:}
Give me the interval between 30 s and 35 s:
all/{30:35}
This applies to any resource, not only logs.
First, put the file in the duckiedown-data-2017 directory on Dropbox.
This is the link.
You need to ask Liam/Andrea because only them have write access.
Then, get the public link address and put it in the file dropbox.urls.yaml in the package easy_node. Remember to have ?dl=1 instead of ?dl=0 in the url.
In the code, use the function require_resource():
from duckietown_utils import require_resource
zipname = require_resource('ii-datasets.zip')
The storage area is in ${DUCKIETOWN_ROOT}/cache/download.
If the file is already downloaded, it is not downloaded again.
(So, if the file changes, you need to delete the cache directory. The best practice is to change the filename every time the file changes.)
The function require_resource() returns the path to the downloaded file.
Also note that you can put any URL in the file dropbox.urls.yaml; but the convention is that we only link things to Dropbox.
easy_nodeeasy_node is a framework to make it easier to create and document ROS nodes.
The main idea is to provide a declarative approach to describe:
The user describes subscriptions, publishers, and parameters in a YAML file.
The framework then automatically takes care of:
Using easy_node allows to cut 40%-50% of the code required for programming
a node. For an example, see the package line_detector2,
which contains a re-implementation of line_detector using the new framework.
Transition plan: The plan is to first use easy_node just for
documenting the nodes. Then, later, convert all the nodes to use it.
For a node with the name my node, implemented in the file
src/my node.py you must create a file by
the name my node.easy_node.yamlsomewhere in the package.
This is the smallest example of an empty configuration:
description: My node
parameters:
subscriptions:
publishers:
contracts:
parameters section: configuring parametersThis is the syntax:
parameters:
name parameter:
type: type
desc: description
default: default value
where:
type is one of float, int, bool, str.description is a description that will appear in the documentation.default gives a default value for the parameter.For example:
parameters:
k_d:
type: float
desc: The derivative gain.
default: 1.02
publishers and subscriptions sectionThe syntax for describing subscribers is:
subscriptions:
name subscription:
topic: topic name
type: message type
desc: description
queue_size: queue size
latch: latch
process: process
The parameters are as follows.
topic name is the name of the topic to subscribe.
message type is a ROS message type name, such as sensor_msgs/Joy.
description is a Markdown description string.
queue size, latch are optional parameters for
ROS publishing/subscribing functions.
See the ROS documentation.
The optional parameter process, one of synchronous (default) or asynchronous describes whether to process the message in a synchronous or asynchronous way (in a separated thread).
The optional parameter timeout describes a timeout value. If no message is received for more than this value, the function on_timeout_subscription() is called.
implement this timeout functionality.
The syntax for describing publishers is similar; it does not have the
the process and timeout value.
Example:
subscriptions:
segment_list:
topic: ~segment_list
type: duckietown_msgs/SegmentList
desc: Line detections
queue_size: 1
timeout: 3
publishers:
lane_pose:
topic: ~lane_pose
type: duckietown_msgs/LanePose
desc: Estimated pose
queue_size: 1
This is not implemented yet.
The idea is to have a place where we can describe constraints such as:
Then, we can implement all these checks once and for all in a proper way, instead of relying on multiple broken implementations
easy_node APIHere is a minimal example of a node that conforms with the API:
from easy_node import EasyNode
class MyNode(EasyNode):
def __init__(self):
EasyNode.__init__(self, 'my_package', 'my_node')
self.info('Initialized.')
if __name__ == '__main__':
MyNode().spin()
The node class must derive from EasyNode.
You need to tell EasyNode what is the package name and the node name.
To initialize, call the function spin().
The EasyNode class provides the following functions:
info()
debug()
error()
These are mapped to rospy.loginfo() etc.; they include the name of the node.
This next example shows how to use configuration parameters.
First, create a file my_node.easy_node.yaml containing:
parameters:
num_cells:
desc: Number of cells.
type: int
default: 42
subscriptions:
contracts:
publishers:
Then, implement the method on_parameters_changed(). It takes
two parameters:
first_time is a boolean that tells whether this is the first time
that the function is called (initialization time).updated is a set of strings that describe the set of parameters
that changed. The first time, it contains the set of all parameters.To access the parameter value, access self.config.parameter.
Example:
class MyNode():
def __init__(self):
EasyNode.__init__(self, 'my_package', 'my_node')
def on_parameters_changed(self, first_time, updated):
if first_time:
self.info('Initializing array for the first time.')
self.cells = [0] * self.config.num_cells
else:
if 'num_cells' in updated:
self.info('Number of cells changed.')
self.cells = [0] * self.config.num_cells
if __name__ == '__main__':
Node().spin()
EasyNode will monitor the ROS parameter server, and will call the function
on_parameters_changed if the user changes any parameters.
To automatically subscribe to topics, add an entry in the subscriptions
section of the YAML file.
For example:
subscriptions:
joy:
desc: |
The `Joy.msg` from `joy_node` of the `joy` package.
The vertical axis of the left stick maps to speed.
The horizontal axis of the right stick maps to steering.
type: sensor_msgs/Joy
topic: ~joy
timeout: 3.0
Then, implement the function on_received_name.
This function will be passed 2 arguments:
context object; this can be used for benchmarking (Section 6.6 - Benchmarking).Example:
class MyNode():
# ...
def on_received_joy(self, context, msg):
# This is called any time a message arrives
self.info('Message received: %s' % msg)
to implement
The function on_timeout_subscription() is called when
there hasnβt been a message for the specified timeout interval.
class MyNode():
# ...
def on_timeout_joy(self, context, time_since):
# This is called when we have not seen a message for a while
self.error('No joystick received since %s s.' % time_since)
The publisher object can be accessed at self.publishers.name.
EasyNode has taken care of all the initialization for you.
For example, suppose we specify a publisher command using:
publishers:
command:
desc: The control command.
type: duckietown_msgs/Twist2DStamped
topic: ~car_cmd
Then we can use it as follows.
class MyNode():
# ...
def on_received_joy(self, context, msg):
out = Twist2DStamped()
out.header.stamp = 0
out.v = 0
out.omega = 0
self.publishers.command.publish(out)
on_init() and on_shutdown()Define the two methods on_init() and on_shutdown() to c
class MyNode(EasyNode):
# ...
def on_init(self):
self.info('Step 1 - Initialized')
def on_parameters_changed(self, first_time, changed):
self.info('Step 2 - Parameters received')
def on_shutdown(self):
self.info('Step 3 - Preparing for shutdown.')
Note that on_init() is called before on_parameters_changed().
easy_node: the userβs point of viewSo far, we have seen how to use parameters from the node, but we did not talk about how to specify the parameters from the userβs point of view.
EasyNode introduces lots of flexibility compared to the legacy system.
The user configuration is specified using files by the pattern
package_name-node_name.config name.config.yaml
where config name is a short string (e.g., baseline).
The files can be anywhere in:
The directory ${DUCKIETOWN_ROOT}/catkin_ws/src;
The directory ${DUCKIEFLEET_ROOT}.
Several config files can exist at the same time. For example, we could have somewhere:
line_detector-line_detector.baseline.config.yaml
line_detector-line_detector.fall2017.config.yaml
line_detector-line_detector.andrea.config.yaml
where the baseline versions are the baseline parameters,
fall2017 are the parameters we are using for Fall 2017,
and andrea are temporary parameters that the user is using.
However, there cannot be two configurations with the same filename
e.g. two copies of line_detector-line_detector.baseline.config.yaml.
In this case, EasyNode will raise an error.
implement this functionality.
The format of the *.config.yaml file is as follows:
description: |
description of what this configuration accomplishes
extends: [config name, config name]
values:
parameter name: value
parameter name: value
The extends field (optional) is a list of string. It allows to use the
specified configurations as the defaults for the current one.
For example, the file line_detector-line_detector.baseline.config.yaml could contain:
description: |
These are the standard values for the line detector.
extends: []
values:
img_size: [120,160]
top_cutoff: 40
Which parameters are used depend on the configuration sequence.
The configuration sequence is a list of configuration names.
It can be specified by the environment variable DUCKIETOWN_CONFIG_SEQUENCE,
using a colon-separated list of strings. For example:
$ export DUCKIETOWN_CONFIG_SEQUENCE=baseline:fall2017:andrea
The line above specifies that the configuration sequence is
baseline, fall2017, andrea.
The system loads the configuration in order. First, it loads the baseline
version. Then it loads the fall2017 version. If a value was already
specified in the baseline version, it is overwritten. If a version does not
exist, it is simply skipped.
If a parameter is not specified in any configuration, an error is raised.
Using this functionality, it is easy to have team-based customization and user-based customization.
There are two special configuration names:
defaultsβ loads the defaults specified by the node.
Note that the defaults are ignored otherwise.vehicleβ expands to the name of the vehicle being used.the vehicle part is not implemented yet.
EasyNode allows to describe configuration that can change in time.
The use case for this is the configuration of calibration parameters:
The solution is to allow a date tag in the configuration name. The format for this is
package_name-node_name.config name.date.config.yaml
For example, we could have the files:
kinematics-kinematics.ferrari.20160404.config.yaml
kinematics-kinematics.ferrari.20170101.config.yaml
Given this, EasyNode will select the configuration to use intelligently. When reading from a bag file from 2016, the first configuration is going to be used; for logs in 2017, the second is going to be used.
not implemented yet.
There are a few tolls used to visualize the configuration database.
easy_node desc: Describing a nodeThe command
$ rosrun easy_node desc package name node name
shows a description of the node, as specified in package name.node name.easy_node.yaml.
For example:
$ rosrun easy_node desc line_detector2 line_detector2
shows the following:
Configuration for node "line_detector_node2" in package "line_detector2"
========================================================================
Parameters
name type default
---- ---- -------
en_update_params_interval float 2.0
top_cutoff int (none)
detector (n/a) (none)
img_size (n/a) (none)
verbose bool True
Subcriptions
name type topic options process
---- ---- ----- ------- -------
switch BoolStamped ~switch queue_size = 1 synchronous
image CompressedImage ~image queue_size = 1 threaded
transform AntiInstagramTransform ~transform queue_size = 1 synchronous
Publishers
name type topic options
---- ---- ----- -------
color_segment Image ~colorSegment queue_size = 1
edge Image ~edge queue_size = 1
segment_list SegmentList ~segment_list queue_size = 1
image_with_lines Image ~image_with_lines queue_size = 1
easy_node eval: Evaluating a configuration sequenceThe program eval script allows to evaluate a certain configuration sequence.
The syntax is:
$ rosrun easy_node eval package name node name config sequence
The tool shows also which file is responsible for the value of each parameter.
For example, the command
$ rosrun easy_node eval line_detector2 line_detector_node2 defaults:andrea
evaluates the configuration for the line_detector_node2 node with the configuration
sequence defaults:andrea.
The result is:
Configuration result for node `line_detector_node2` (package `line_detector2`)
The configuration sequence was ['defaults', 'baseline', 'andrea'].
The following is the list of parameters set and their origin:
parameter value origin
--------- ----- ------
en_update_params_interval 2.0 defaults
top_cutoff 40 baseline
detector - line_detector.LineDetectorHSV baseline
- configuration:
canny_thresholds: [80, 200]
dilation_kernel_size: 3
hough_max_line_gap: 1
hough_min_line_length: 3
hough_threshold: 2
hsv_red1: [0, 140, 100]
hsv_red2: [15, 255, 255]
hsv_red3: [165, 140, 100]
hsv_red4: [180, 255, 255]
hsv_white1: [0, 0, 150]
hsv_white2: [180, 60, 255]
hsv_yellow1: [25, 140, 100]
hsv_yellow2: [45, 255, 255]
img_size [120, 160] baseline
verbose true defaults
Note how we can tell which configuration file is responsible for setting each parameter.
easy_node summary: Describing and validating all configuration filesThe program summary reads all the configuration files described in the repository
and validates them against the node description.
If a configuration file refers to parameters that do not exist, the configuration file is established to be invalid.
The syntax is:
$ rosrun easy_node summary
For example, the output can be:
package name node name config_name effective extends valid error description
line_detector2 line_detector_node2 baseline 2017-01-01 () yes These are the standard values for t [..]
EasyNode implements some simple timing statistics. These are accessed using
the context object passed to the message received callbacks.
Hereβs an example use, from line_detector2:
def on_received_image(self, context, image_msg):
with context.phase('decoding'):
...
with context.phase('resizing'):
# Resize and crop image
...
stats = context.get_stats()
self.info(stats)
The idea is to enclose the different phases of the computation
using the context manager phase(name).
A summary of the statistics can be accessed by using context.get_stats().
For example, this will print:
Last 24.4 s: received 734 (30.0 fps) processed 301 (12.3 fps) skipped 433 (17.7 fps) (59 %)
decoding | total latency 25.5 ms | delta wall 20.7 ms | delta clock 20.7 ms
resizing | total latency 26.6 ms | delta wall 0.8 ms | delta clock 0.7 ms
correcting | total latency 29.1 ms | delta wall 2.2 ms | delta clock 2.2 ms
detection | total latency 47.7 ms | delta wall 18.2 ms | delta clock 21.3 ms
preparing-images | total latency 55.0 ms | delta wall 7.0 ms | delta clock 7.0 ms
publishing | total latency 55.5 ms | delta wall 0.1 ms | delta clock 0.1 ms
draw-lines | total latency 59.7 ms | delta wall 4.0 ms | delta clock 3.9 ms
published-images | total latency 61.2 ms | delta wall 0.9 ms | delta clock 0.8 ms
pub_edge/pub_segment | total latency 86.3 ms | delta wall 24.7 ms | delta clock 24.0 ms
EasyNode generated the documentation automatically from the *.easy_node.yaml
files.
Note that this can be used independently of the fact that the node
implements the EasyNode API. So, we can use this EasyNode functionality
also to document the legacy nodes.
To generate the docs, use this command:
$ rosrun easy_node generate_docs
For each node documented using a *.easy_node.yaml, this generates a Markdown
file called βnode name-easy_node-autogenerated.mdβ in the package
directory.
The contents are enclosed in a div with ID #package name-node name-autogenerated.
The intended use is that this can be used to move the contents
to the place in the documentation using the move-here tag.
For example, in the README.md of the joy_mapper package, we have:
## Node `joy_mapper_node`
<move-here src="#joy_mapper-joy_mapper_node-autogenerated"/>
The result can be seen at Unit P-3 - Package joy_mapper.
(Generated from configuration easy_node.easy_node.yaml.)
These are properties common to all nodes.
No parameters defined.
No subscriptions defined.
No publishers defined.
(Add here other ideas that we can implement.)
easy_regressionto write
A regression test is defined by the following:
Letβs go more in detail, to obtain a formalization that can be used to parallelize the processing.
This is in the spirit of a map/reduce framework.
Map-reduce framework XXX
We assume that a processor has the following signature:
$$ \text{processor}: \text{Bag} \rightarrow \text{Bag} $$
An analyzer is something that returns a statistics of type Stat:
$$ \text{analyzer}: \text{Bag} \rightarrow \text{Stat} $$
We also assume to have an operation merge that allows to merge
the results of two statistics:
$$ \text{merge}: \text{Stat} \times \text{Stat} \rightarrow \text{Stat} $$
Finally, the acceptance condition is a test on the statistics. We want to be careful about the possible conditions, so instead of having only true/false, we give the following results:
$$ \text{acceptance}: \text{Stat} \rightarrow \{ \texttt{ok}, \texttt{fail}, \texttt{nodata}, \texttt{abnormal}\} $$
The semantics is as follows:
ok means that the conditions is satisfied;fail means that the condition is not satisfied;nodata means that there is not enough data to check the condition. For example, there are conditions that reference the results of regression tests at another date; that data
might be not available;abnormal covers all abnormal conditions, including, for example, mistakes in writing
the testing conditions.The regression test engine does the following.
It reads the log test. It makes the logs available. (This might imply downloading the logs.)
A physical log is a physical .bag file. These are generated
regression1.regression_test.yaml
description:
Simple regression test.
a
logs:
- "vehicle:ferrari,length:<10"
processors:
- transformer: Identity
stats:
visualizers:
identity.job_processors.yaml
description:
constructor: IdentityProcessor
parameters:
count:
S is a dict of YAML
processor: Bag -> Bag
analyzer: Bag -> S
reduce: S x S -> S
display_test: S -> Display
A regression test is characterized by
Using an EasyAlgo configuration file, we can describe
a simple regression tests that counts the number of messages in a bag
in a file example.regression_test.yaml as follows:
description: Simple regression test
constructor: easy_regression.RegressionTest
parameters:
logs:
- 20160223-amadoa-amadobot-RCDP2
processors: []
analyzers:
- count_messages
checks:
- desc: The number of messages read should remain the same.
cond: |
v:count_messages/20160223-amadoa-amadobot-RCDP2/num_messages == 5330
In this example, there is only one log, no processors, and one analyzer.
There is one condition, that checks that the output of count_messages on that log is 5330.
v:count_messages/20160223-amadoa-amadobot-RCDP2/num_messages == 5330
We can run this regression test using:
$ rosrun easy_regression run --tests example
There is a flexible language that allows to check conditions.
The simplest checks regarding checking the values:
v:analyzer/log/statistics == value
v:analyzer/log/statistics >= value
v:analyzer/log/statistics <= value
v:analyzer/log/statistics < value
v:analyzer/log/statistics > value
Check that it is in 10% of the value:
v:analyzer/log/statistics ==[10%] value
Use @date to reference the test result at that date.
For example,
v:analyzer/log/statistics ==[10%] v:analyzer/log/statistic@2017-08-01
Use ~branch@date to reference the value of a branch at a certain date:
v:analyzer/log/statistics ==[10%] v:analyzer/log/statistic~master@2017-08-01
Use ?commit to reference the value of a branch at a specific commit:
v:analyzer/log/statistics ==[10%] v:analyzer/log/statistic?e9aa5f4
what_the_duckwhat-the-duck is a program that tests dozens of configuration
inconsistencies that can happen on a Duckiebot.
what-the-duck programThe proper usage of what-the-duck to debug an environment problem is the following
sequence:
$ # open a new terminal
$ cd Duckietown root
$ git checkout master
$ git pull
$ source environment.sh
$ ./dependencies_for_duckiebot.sh # if you are on a Duckiebot
$ ./dependencies_for_laptop.sh # if you are on a laptop
$ make build-clean
$ make build-catkin
$ ./what-the-duck
you have to do all the steps in the precise order.
The telemetry is collected and available at this URL.
to write
adafruit_driversAdd a description of package adafruit_drivers in package.xml.
These are the Adafruit drivers.
What is the original location of this package?
dagu_carforward_kinematics_node(Generated from configuration forward_kinematics_node.easy_node.yaml.)
Missing node description in forward_kinematics_node.easy_node.yaml.
No parameters defined.
No subscriptions defined.
No publishers defined.
inverse_kinematics_node(Generated from configuration inverse_kinematics_node.easy_node.yaml.)
Missing node description in inverse_kinematics_node.easy_node.yaml.
No parameters defined.
No subscriptions defined.
No publishers defined.
velocity_to_pose_node(Generated from configuration velocity_to_pose_node.easy_node.yaml.)
Missing node description in velocity_to_pose_node.easy_node.yaml.
No parameters defined.
No subscriptions defined.
No publishers defined.
car_cmd_switch_node(Generated from configuration car_cmd_switch_node.easy_node.yaml.)
Missing node description in car_cmd_switch_node.easy_node.yaml.
No parameters defined.
No subscriptions defined.
No publishers defined.
wheels_driver_node(Generated from configuration wheels_driver_node.easy_node.yaml.)
Missing node description in wheels_driver_node.easy_node.yaml.
No parameters defined.
No subscriptions defined.
No publishers defined.
wheels_trimmer_node(Generated from configuration wheels_trimmer_node.easy_node.yaml.)
Missing node description in wheels_trimmer_node.easy_node.yaml.
No parameters defined.
No subscriptions defined.
No publishers defined.
joy_mapperConnect joystick.
Run:
$ roslaunch launch/joy_mapper_test.launch
The robot should move when you push buttons.
rospysensor_msgs: for the Joy.msgduckietown_msgs: for the CarControl.msgjoy_mapper_node2(Generated from configuration joy_mapper_node2.easy_node.yaml.)
This node takes a sensor_msgs/Joy.msg and converts it to a
duckietown_msgs/CarControl.msg.
It publishes at a fixed interval with a zero-order hold.
Parameter v_gain: float; default value: 0.41
Missing description for entry βv_gainβ.
Parameter omega_gain: float; default value: 8.3
Missing description for entry βomega_gainβ.
Parameter bicycle_kinematics: int; default value: 0
Missing description for entry βbicycle_kinematicsβ.
Parameter steer_angle_gain: int; default value: 1
Missing description for entry βsteer_angle_gainβ.
Parameter simulated_vehicle_length: float; default value: 0.18
Missing description for entry βsimulated_vehicle_lengthβ.
Subscription joy: topic joy (Joy)
The Joy.msg from joy_node of the joy package. The vertical axis of the left stick maps to speed. The horizontal axis of the right stick maps to steering.
Publisher car_cmd: topic ~car_cmd (Twist2DStamped)
Missing description for entry βcar_cmdβ.
Publisher joy_override: topic ~joystick_override (BoolStamped)
Missing description for entry βjoy_overrideβ.
Publisher parallel_autonomy: topic ~parallel_autonomy (BoolStamped)
Missing description for entry βparallel_autonomyβ.
Publisher anti_instagram: topic anti_instagram_node/click (BoolStamped)
Missing description for entry βanti_instagramβ.
Publisher e_stop: topic wheels_driver_node/emergency_stop (BoolStamped)
Missing description for entry βe_stopβ.
Publisher avoidance: topic ~start_avoidance (BoolStamped)
Missing description for entry βavoidanceβ.
pi_cameracamera_node_sequence(Generated from configuration camera_node_sequence.easy_node.yaml.)
Camera driver, second approach.
No parameters defined.
No subscriptions defined.
No publishers defined.
camera_node_continuous(Generated from configuration camera_node_continuous.easy_node.yaml.)
Camera driver.
Parameter framerate: float; default value: 60.0
Frame rate
Parameter res_w: int; default value: 320
Resolution (width)
Parameter res_h: int; default value: 200
Resolution (height)
No subscriptions defined.
Publisher image_compressed: topic ~image/compressed" (CompressedImage)
Missing description for entry βimage_compressedβ.
decoder_node(Generated from configuration decoder_node.easy_node.yaml.)
A node that decodes a compressed image into a regular image. This is useful so that multiple nodes that need to use the image do not do redundant computaon.
Parameter publish_freq: float; default value: 1.0
Frequency at which to publish (Hz).
Subscription compressed_image: topic ~compressed_image (CompressedImage)
The image to decode.
Subscription switch: topic ~switch (BoolStamped)
Switch to turn on or off. The node starts as active.
Publisher raw: topic ~image/raw (Image)
The decoded image.
img_process_node(Generated from configuration img_process_node.easy_node.yaml.)
Apparently, a template, or a node never finished.
No parameters defined.
No subscriptions defined.
No publishers defined.
cam_info_reader_node(Generated from configuration cam_info_reader_node.easy_node.yaml.)
Publishes a CameraInfo message every time it receives an image.
Parameter config: str; default value: 'baseline'
Missing description for entry βconfigβ.
Parameter cali_file_name: str; default value: 'default'
Missing description for entry βcali_file_nameβ.
Parameter image_type: str; default value: 'compressed'
Missing description for entry βimage_typeβ.
Subscription compressed_image: topic ~compressed_image (CompressedImage)
If image_type is βcompressedβ then itβs CompressedImage, otherwise Image.
Publisher camera_info: topic ~camera_info (CameraInfo)
Missing description for entry βcamera_infoβ.
to write
anti_instagramrostestUnit tests are integrated with rostest.
To run manually, use:
$ rostest anti_instagram antiinstagram_correctness_test.test
$ rostest anti_instagram antiinstagram_stub_test.test
$ rostest anti_instagram antiinstagram_performance_test.test
These are other unittest that require the logs in DUCKIETOWN_DATA:
$ rosrun anti_instagram annotations_test.py
anti_instagram_node(Generated from configuration anti_instagram_node.easy_node.yaml.)
Missing node description in anti_instagram_node.easy_node.yaml.
Parameter publish_corrected_image: bool; default value: False
Whether to compute and publish the corrected image.
Subscription image: topic ~uncorrected_image (CompressedImage)
This is the compressed image to read.
Subscription click: topic ~click (BoolStamped)
Activate the calibration phase with this switch.
Publisher image: topic ~corrected_image (Image)
The corrected image.
Publisher health: topic ~colorSegment (AntiInstagramHealth)
The health of the process.
Publisher transform: topic ~transform (AntiInstagramTransform)
The computed transform.
complete_image_pipelinesingle_image_pipelineThis runs the entire pipeline:
$ rosrun complete_image_pipeline single_image_pipeline --image image --line_detector detector --image_pre image_prep
where image can be a filename or a URL.
For example, this:
$ rosrun complete_image_pipeline single_image_pipeline --image "https://www.dropbox.com/s/bzezpw8ivlfu4b0/frame0002.jpg?dl=1"
results in an output as in Figure 2.1.
single_image_pipelineground_projectionground_projection_node(Generated from configuration ground_projection.easy_node.yaml.)
node description for ground_projection
No parameters defined.
No subscriptions defined.
No publishers defined.
lane_controllane_controller_node(Generated from configuration lane_controller_node.easy_node.yaml.)
there is some very funny business inside. It appears that k_d and k_theta are switched around.
Parameter v_bar: float
Nominal linear velocity (m/s).
Parameter k_theta: float
Proportional gain for $\theta$.
Parameter k_d: float
Propertional gain for $d$.
Parameter d_thres: float
Cap for error in $d$.
Parameter theta_thres: float
Maximum desired $\theta$.
Parameter d_offset: float
A configurable offset from the lane position.
Subscription lane_reading: topic ~lane_pose (LanePose)
Missing description for entry βlane_readingβ.
Publisher car_cmd: topic ~car_cmd (Twist2DStamped)
Missing description for entry βcar_cmdβ.
lane_filterLiam
lane_filter_node(Generated from configuration lane_filter_node.easy_node.yaml.)
Missing node description in lane_filter_node.easy_node.yaml.
Parameter mean_d_0: float; default value: 0.0
Missing description for entry βmean_d_0β.
Parameter mean_phi_0: float; default value: 0.0
Missing description for entry βmean_phi_0β.
Parameter sigma_d_0: float; default value: 0.0
Missing description for entry βsigma_d_0β.
Parameter sigma_phi_0: float; default value: 0.0
Missing description for entry βsigma_phi_0β.
Parameter delta_d: float; default value: 0.02
(meters)
Parameter delta_phi: float; default value: 0.0
(radians)
Parameter d_max: float; default value: 0.5
Missing description for entry βd_maxβ.
Parameter d_min: float; default value: -0.7
Missing description for entry βd_minβ.
Parameter phi_min: float; default value: -1.5707
Missing description for entry βphi_minβ.
Parameter phi_max: float; default value: 1.5707
Missing description for entry βphi_maxβ.
Parameter cov_v: float; default value: 0.5
Linear velocity βinputβ.
XXX which units?
Parameter cov_omega: float; default value: 0.01
Angular velocity βinputβ.
XXX which units?
Parameter linewidth_white: float; default value: 0.04
Missing description for entry βlinewidth_whiteβ.
Parameter linewidth_yellow: float; default value: 0.02
Missing description for entry βlinewidth_yellowβ.
Parameter lanewidth: float; default value: 0.4
Missing description for entry βlanewidthβ.
Parameter min_max: float; default value: 0.3
Expressed in nats.
Parameter use_distance_weighting: bool; default value: False
For use of distance weighting (dw) function.
Parameter zero_val: float; default value: 1.0
Missing description for entry βzero_valβ.
Parameter l_peak: float; default value: 1.0
Missing description for entry βl_peakβ.
Parameter peak_val: float; default value: 10.0
Missing description for entry βpeak_valβ.
Parameter l_max: float; default value: 2.0
Missing description for entry βl_maxβ.
Parameter use_max_segment_dist: bool; default value: False
For use of maximum segment distance.
Parameter max_segment_dist: float; default value: 1.0
For use of maximum segment distance.
Parameter use_min_segs: bool; default value: False
For use of minimum segment count.
Parameter min_segs: int; default value: 10
For use of minimum segment count.
Parameter use_propagation: bool; default value: False
For propagation.
Parameter sigma_d_mask: float; default value: 0.05
Missing description for entry βsigma_d_maskβ.
Parameter sigma_phi_mask: float; default value: 0.05
Missing description for entry βsigma_phi_maskβ.
Subscription velocity: topic ~velocity (Twist2DStamped)
Missing description for entry βvelocityβ.
Subscription segment_list: topic ~segment_list (SegmentList)
Missing description for entry βsegment_listβ.
Publisher lane_pose: topic ~lane_pose (LanePose)
Missing description for entry βlane_poseβ.
Publisher belief_img: topic ~belief_img (Image)
Missing description for entry βbelief_imgβ.
Publisher entropy: topic ~entropy (Float32)
Missing description for entry βentropyβ.
Publisher in_lane: topic ~in_lane (BoolStamped)
Missing description for entry βin_laneβ.
Publisher switch: topic ~switch (BoolStamped)
Missing description for entry βswitchβ.
line_detector2This is a re-implementation of the package line_detector
using the new facilities provided by easy_node.
The following are instructions to test the line detector from bag files.
You can run from a bag with the following:
laptop $ roslaunch line_detector line_detector_bag.launch veh:=vehicle bagin:=bag in bagout:=bag out verbose:=true
Where:
bag in is the absolute path of the input bag.vehicle is the name of the vehicle that took the log.bag out is the absolute path if the output bag.you always need to use absolute paths for bag files.
You can let this run for a few seconds, then stop using Ctrl-C.
You can then inspect the result using:
$ roscore &
$ rosbag play -l bag out
$ rviz &
In rviz click βaddβ, click βby topicβ tab, expand βline_detectorβ and click
βimage_with_linesβ.
Observe on the result that:
These are some sample logs on which to try:
$ wget -O 160122-manual1_ferrari.bag https://www.dropbox.com/s/8bpi656j7qox5kv?dl=1
https://www.dropbox.com/s/vwznjke4xvnhi9o/160122_manual2-ferrari.bag?dl=1
https://www.dropbox.com/s/y7uljl98punj0mp/160122_manual3_corner-ferrari.bag?dl=1
https://www.dropbox.com/s/d4n9otmlans4i62/160122-calibration-good_lighting-tesla.bag?dl=1
Sample output:
From cmd:roslaunch duckietown camera.launch veh:=${VEHICLE_NAME}
[INFO] [WallTime: 1453839555.948481] [LineDetectorNode] number of white lines = 14
[INFO] [WallTime: 1453839555.949102] [LineDetectorNode] number of yellow lines = 33
[INFO] [WallTime: 1453839555.986520] [LineDetectorNode] number of white lines = 18
[INFO] [WallTime: 1453839555.987039] [LineDetectorNode] number of yellow lines = 34
[INFO] [WallTime: 1453839556.013252] [LineDetectorNode] number of white lines = 14
[INFO] [WallTime: 1453839556.013857] [LineDetectorNode] number of yellow lines = 29
[INFO] [WallTime: 1453839556.014539] [LineDetectorNode] number of red lines = 2
[INFO] [WallTime: 1453839556.047944] [LineDetectorNode] number of white lines = 18
[INFO] [WallTime: 1453839556.048672] [LineDetectorNode] number of yellow lines = 28
[INFO] [WallTime: 1453839556.049534] [LineDetectorNode] number of red lines = 2
[INFO] [WallTime: 1453839556.081400] [LineDetectorNode] number of white lines = 13
[INFO] [WallTime: 1453839556.081944] [LineDetectorNode] number of yellow lines = 34
[INFO] [WallTime: 1453839556.082479] [LineDetectorNode] number of red lines = 1
The output from rviz looks like Figure 6.1.
Something more quantitative (to be filled in by Liam or Hang)
line_detector_node2(Generated from configuration line_detector_node2.easy_node.yaml.)
This is a rewriting of line_detector_node using the EasyNode framework.
Parameter verbose: bool; default value: True
Whether the node is verbose or not. If set to True,
the node will write timing statistics to the log.
Parameter img_size: not known
Missing description for entry βimg_sizeβ.
Parameter top_cutoff: int
This parameter decides how much of the image we should cut off. This is a performance improvement.
Parameter line_detector: str
This is the instance of line_detector to use.
Subscription image: topic ~image (CompressedImage)
This is the compressed image to read. Note that it takes a long time to simply decode the image JPG.
The data is processed asynchronously in a different thread.
Subscription transform: topic ~transform (AntiInstagramTransform)
The anti-instagram transform to apply. See Unit Q-1 - Package anti_instagram.
Subscription switch: topic ~switch (BoolStamped)
This is a switch that allows to control the activity of this node. If the message is true, the node becomes active. If false, it switches off. The node starts as active.
Publisher edge: topic ~edge (Image)
Missing description for entry βedgeβ.
Publisher color_segment: topic ~colorSegment (Image)
Missing description for entry βcolor_segmentβ.
Publisher segment_list: topic ~segment_list (SegmentList)
Missing description for entry βsegment_listβ.
Publisher image_with_lines: topic ~image_with_lines (Image)
Missing description for entry βimage_with_linesβ.
line_detectorAdd a description of package line_detector in package.xml.
This package is being replaced by the cleaned-up version, line_detector2. Do not write documentation here.
However, at the moment, all the launch files still call this one.
to write
duckietown_descriptionlocalizationto write
led_detectionPick your favourite duckiebot as the observer-bot. Refer to it as robot name for this step. If you are in good company, this can be tried on all the available duckiebots. First, activate the camera on the observer-bot:
$ roslaunch duckietown camera.launch veh:=robot name
In a separate terminal, fire up the LED detector and the custom GUI by running:
$ roslaunch led_detector LED_detector_with_gui.launch veh:=robot name
to operate without a GUI:
laptop $ roslaunch led_detector LED_detector.launch veh:=robot name
The LED_detector_node will be launched on the robot, while LED_visualizer (a simple GUI) will be started on your laptop. Make sure the camera image from the observer-bot is visualized and updated in the visualizer (tip: check that your camera cap is off).
Hit on Detect and wait to trigger a detection. This will not have any effect if LED_detector_node is not running on the duckiebot (it is included in the above launch file). After the capture and processing phases, the outcome will look like:
The red numbers represent the frequencies directly inferred from the camera stream, while the selected detections with the associated signaling frequencies will be displayed in green. You can click on the squares to visualize the brightness signals and the Fourier amplitude spectra of the corresponding cells in the video stream. You can also click on the camera image to visualize the variance map.
To run the unit tests for the LED detector, you need to have the F23 rosbags on you hard disk. These bag files should be synced from [this dropbox link] (https://www.dropbox.com/sh/5kx8qwgttu69fhr/AAASLpOVjV5r1xpzeW7xWZh_a?dl=0).
For the test to locate the bag files, you should have the DUCKIETOWN_DATA environment variable set, pointing to the location of you duckietown-data folder. This can be achieved by:
$ export DUCKIETOWN_DATA=local-path-to-duckietown-data-folder
All the available tests are specified in file all_tests.yaml in the scripts/ folder of the package led_detection in the duckietown ROS workspace. To run these, use the command:
$ rosrun led_detection unittests.py algorithm name-of-test
Currently, algorithm can be either βbaselineβ or βLEDDetector_plotsβ to also display the plot in the process.
To run all test with all algorithms, execute:
$ rosrun led_detection unittests.py '*' '*'
More in general:
$ rosrun led_detection unittests.py tests algorithms
where:
tests is a comma separated list of algorithms. May use β*β.algorithms is a comma separated list of algorithms. May use β*β.The default algorithm is called βbaselineβ, and its tests are invoked using:
$ rosrun led_detection <script> '*' 'baseline'
led_emitterThe coordination team will use 3 signals: CAR_SIGNAL_A, CAR_SIGNAL_B, CAR_SIGNAL_C. To test the LED emitter with your joystick, run the following command:
$ roslaunch led_joy_mapper led_joy_with_led_emitter_test.launch veh:=robot name
This launches the joy controller, the mapper controller, and the led emitter nodes. You should not need to run anything external for this to work. Use the joystick buttons A, B and C to change your duckiebotβs LEDβs blinking frequency.
Button A broadcasts signal CAR_SIGNAL_A (2.8hz), button B broadcasts signal CAR_SIGNAL_B (4.1hz), and button CAR_SIGNAL_C (Y on the controller) broadcasts signal C(5hz).The LB button will make the LEDs all white, the RB button will make some LEDs blue and some LEDs green, and the logitek button (middle button) will make the LEDs all red
Repeat this for each vehicle at the intersection that you wish to be blinking. Use previous command replacing robot name the names of the vehicles and try command different blinking patterns on different duckiebots.
(optional tests) For a grasp of the low level LED emitter, run:
$ roslaunch led_emitter led_emitter_node.launch veh:=robot name
You can then publish to the topic manually by running the following command in another screen on the duckiebot:
$ rostopic pub /robot name/led_emitter_node/change_to_state std_msgs/Float32 float-value
Where float-value is the desired blinking frequency, e.g. 1.0, .5, 3.0, etc. If you wish to run the LED emitter test, run the following:
$ roslaunch led_emitter led_emitter_node_test.launch veh:=robot name
This will cycle through frequencies of 3.0hz, 3.5hz, and 4hz every 5 seconds. Once done, kill everything and make sure you have joystick control as described above.
led_interpreterled_joy_mapperrgb_ledTo test the traffic light:
$ rosrun rgb_led blink trafficlight4way
Fancy test:
$ rosrun rgb_led blink trafficlight4way
To do other tests:
$ rosrun rgb_led blink
traffic_lightto write
mdoapparallel_autonomyvehicle_detectionThese are templates.
pkg_nameThe package pkg_name is a template for ROS packages.
For the tutorial, see Unit K-6 - Minimal ROS node - pkg_name.
This package is a template that contains code
rostest_exampleThese packages are convenience packages that group together launch files and tests.
duckie_rr_bridgeduckiebot_visualizerduckiebot_visualizer(Generated from configuration duckiebot_visualizer.easy_node.yaml.)
Missing node description in duckiebot_visualizer.easy_node.yaml.
Parameter veh_name: str; default value: 'megaman'
Missing description for entry βveh_nameβ.
Subscription seg_list: topic ~segment_list (SegmentList)
Missing description for entry βseg_listβ.
Publisher pub_seg_list: topic ~segment_list_markers (MarkerArray)
Missing description for entry βpub_seg_listβ.
duckietown_demosduckietown_logsUntil we fix the dependencies:
sudo pip install SystemCmd==1.2 ros_node_utils==1.0 ConfTools==1.8 QuickApp==1.2.2
sudo apt-get install -y mplayer mencoder
sudo add-apt-repository ppa:mc3man/trusty-media
sudo apt-get update
sudo apt-get install -y ffmpeg gstreamer0.10-ffmpeg
duckietown_unit_testveh_coordinatorI think is used to fake the vehicle coordination for the FSM.
DB17-jwd) (liampaull)DB17-wjd TTIC) (Andrea F. Daniele)DB17) (Andrea F. Daniele)DB17) (Andrea F. Daniele)DB17-jwd) (Andrea F. Daniele)DB17-lc configurations (Andrea F. Daniele)DB17-lc) (Andrea F. Daniele)DB17-l) (Andrea F. Daniele)DB17-lc) (Andrea F. Daniele)DB17-l setup (Andrea F. Daniele)line_detector2 (Philippe Laferrière)easy_logs (Andrea Censi)what_the_duck (Andrea Censi)pkg_name (Andrea F. Daniele)
comment
this is a comment on one paragraph.